Home

Oracle ZFS Storage Appliance Customer Service Manual

1. prs zm fig 8 Six HBAs and 36 disk shelves in six chains 186 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 2 7120 7320 with DE2 and Sun Disk Shelves 2S3 2 7120 7320 with DE2 and Sun Disk Shelves Connecting ZS3 2 7120 7320 Standalone Controllers to Mixed Disk Shelves The following figures show a subset of the supported configurations for Oracle ZFS Storage ZS3 2 7120 7320 standalone controllers with one HBA Using mixed disk shelves on a controller requires the following The controller must use only 4X4 port SAS 2 HBAs Do not use mixed disk shelves in the same chain Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model 4X4 port SAS 2 HBAs are only supported with release AK 2013 1 0 and later Cable Diagrams fig 1Standalone controller with one HBA and two disk shelves in two chains DE2 24 on the left 3 3 EST m mec ens 9 3 gare De anu 3 3 3 3 Lata fig 2 Eight disk shelves in two chains DE2 24 on the left Chapter 2 Hardware Maintenance 187 ZS3 2 7320 Clustered with DE2 and Sun Disk Shelves 2S3 2 7320 Clustered with DE2 and Sun Disk Shelves Connecting ZS3 2 7320 Storage Clustered Controllers to Mixed Disk Shelves The following figures show a subset of the supported configuratio
2. Factory Reset rines aa cd e qoa a de deoa ee a a d d d a de e acea Updates race net dedu ente dicia auateqe diia wanted System Updates Hardware Firmware Updates oes eite e rtt tee erede 204 olco A 206 Cluster Uperade TEE 208 Updating Vid the BUI ii cete ei eene ed tinent t dete id ate de 209 Updating via the GL rte teca enda dur ania aan ge tates 211 Passthrough x Passthrough x Deferred Update eterne ttti been dee o iei eae rh Ur QUOLdSi carretera te trad ed NER D OE RUSO tr rere ert EI CELO RD eie ere Serene User Quotas Deferred Update estet eniin ette mise bereit rises COMIS TAR 3 cieudecpere quce ti nica eei en ictu iiia denied d fico osadia seuls COMSTAR Deferred Update aderire ctii etsi bet etri stes ete icp ES Triple Parity RAID Triple Parity RAID Deferred Update Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Contents Data Deduplication Deferred Update RepliGatiOn ND Si Replication Deferred Update octo tasa to nibeaw NU EH dirt reads Received PFODErtles code gebe DR RERO ED RES ibn EUR ERU Received Properties Deferred Update Slim Zl a A moii eae Ue debere da cde EIU TM trOGUctiOn MR Snapshot Deletion e aces cabanas adi i ieetan ddr edad verat Snapshot Deletion Deferred Update eee ein eer tieni Recursive Snapshots noise dod e e e e o p ed Fk v ed e oe a i Recu
3. fig 1 Clustered controllers with two HBAs and multiple mixed disk shelves in two chains DE2 24 on the left Chapter 2 Hardware Maintenance 189 ZS3 4 7420 Clustered with DE2 and Sun Disk Shelves fig 2 Eight disk shelves in two chains DE2 24 on the left ZS3 4 7420 Clustered with DE2 and Sun Disk Shelves Connecting ZS3 4 7420 Clustered Controllers to Mixed Disk Shelves The following figures show a subset of the supported configurations for Oracle ZFS Storage ZS3 4 7420 clustered controllers with two HBAs Using mixed disk shelves on a controller requires the following The controller must use only 4X4 port SAS 2 HBAs Do not use mixed disk shelves in the same chain Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model 4X4 port SAS 2 HBAs are only supported with release AK 2013 1 0 and later Cable Diagrams 190 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 7420 Clustered with DE2 and Sun Disk Shelves fig 1Clustered controllers with two HBAs and multiple mixed disk shelves in two chains DE2 24 on the left fig 2 Eight disk shelves in two chains DE2 24 on the left Chapter2 Hardware Maintenance 191 192 System CHAPTER 3 System Maintenance Introduction The Maintenance System screen provides several system level fe
4. Maintenance Introduction This section describes concepts and procedural instructions for performing hardware and software maintenance tasks The graphic above illustrates locating a spare disk within the chassis by highlighting its name in the BUI Hardware Maintenance list The Maintenance gt Hardware screen of the BUI provides visual representations of the physical system components allowing you to visually identify and locate hardware components and verify their status Software Updates can be applied in the System section of the interface as well as viewing Logs and current Problems Hardware Overview identify hardware components and verify their status Controllers 27 253 2 253 2 28 m ZS3 2Overview component diagrams and specifications ma 2Z S3 2Maintenance Procedures replace controller drives fans power supplies memory cards and batteries ZS3 4 Overview component diagrams and specifications m ZS3 4 Maintenance Procedures replace controller drives fans power supplies memory cards risers and batteries 7120 7320 7420 Overviews component diagrams and specifications ma 7x20 Maintenance Procedures replace controller drives fans power supplies memory cards risers and batteries Expansion Storage Disk Shelf Overview component diagrams and specifications for Oracle Storage Drive Enclosure DE2 24 and Sun Disk Shelf Disk Shelf Maintenance Procedures r
5. Remove the Top Cover and Air Filter To access internal controller components for servicing you must remove the top cover 1 Fully loosen the two captive screws at the rear ofthe top cover 1 54 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 2 Slidethetop cover rearward 0 5 in 13 mm and lift it straight up and offthe chassis 2 and set the top cover aside 3 Pinch theleft and right release levers inward 3 4 Pivottheair filter forward and lift if out of the chassis 4 Chapter2 Hardware Maintenance 55 253 2 Controller Replacement Tasks ZS3 2 Controller Replacement Tasks SSDs and HDDs l Identify the failed drive by going to the Maintenance Hardware section ofthe BUI and clicking the drive details icon amp If you are physically at the system the amber Service Required indicator on the HDD or SSD should be illuminated 2 Ifyouare not physically at the system turn on the locator indicator by clicking the locator icon amp 3 Pressthe release button and pivot the release lever open to the right 5 After 15 seconds navigate to the Hardware Maintenance screen and click the details icon on the system controller to verify that the software has detected that the drive is not present 6 Alignthe replacement drive with the drive slot 56 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 7 P
6. At the top right of the system chassis is the locate icon X to light the locate LED and the power icon which presents a dialog box to either power off or reboot power cycle the appliance A thumbnail of the controller is presented at left Clicking on the thumbnail or the Show Details link takes you to a detailed view of the chassis and is identical to clicking on the right pointing arrow at the top left of the view Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Hardware The following information is presented in a summary view Property Description Manufacturer Manufacturer ofthe system Model System model name Serial System chassis hardware serial number Processors Count and description of processors in the system Memory Total memory in the system System Size and number of system disks used for the system image Data Size and number of data disks in the system chassis This is only valid for standalone systems If there are no data disks present will be displayed Cache Size and number of cache disks in the system chassis This is only valid for expandable systems that support additional disk shelves If there are no cache disks present will be displayed Log Size and number of log disks in the system chassis This is only valid for standalone systems If there are no log devices present will be displayed Total Total size and count of
7. Enabled datalink igb0 service annotation The annotation is the session annotation which can be enabled when configuring users Chapter 3 System Maintenance 229 230 7120 7320 7420 Active Directory Alerts Analytics ARC BUI CLI Cluster Controller CPU CRU Dashboard Dataset DIMM Disk Shelf DNS DTrace FC FRU FTP Glossary Sun ZFS Storage 7120 Sun ZFS Storage 7320 Sun ZFS Storage 7420 Microsoft Active Directory server Configurable log email or SNMP trap events appliance feature for graphing real time and historic performance statistics Adaptive Replacement Cache Browser User Interface Command Line Interface Multiple heads connected to shared storage See Storage Controller Central Prcessing Unit Customer Replaceable Component appliance summary display of system health and activity the in memory and on disk data for a statistic from Analytics dual in line memory module the expansion storage shelf that is connected to the head node or storage controller Domain Name Service a comprehensive dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time Fibre Channel Field Replaceable Component File Transfer Protocol 231 GigE GigE HBA HCA HDD HTTP Hybrid Storage Pool Icons IOM iSCSI Kiosk L2ARC LDAP LED Logzilla LUN Masthead Modal Dialog NFS NIC NIS PCIe PCM Pool Project PSU QDR Readzi
8. 9 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 10 Connect the power cords to the power supplies 11 Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in 12 Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit 13 Connect data cables to the PCIe card routing them through the cable management arm 14 Goto the Maintenance Hardware screen of the BUI and click the details icon amp on the controller Then click Slots to verify the status ofthe new component The status indicator should appear green 15 Install the disk shelf and connect the expansion storage Battery You might need to replace the battery if the storage controller fails to maintain the proper time when powered off and disconnected from the network You will need a small No 1 flat blade non metallic screwdriver or equivalent You must shut down the appliance before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the
9. Cabling 170 Connecting to Attached Storage The cabling information in this section shows a subset of the supported configurations for Oracle ZFS Storage controllers For the maximum number of disk shelves per controller see the Summary of Controller Configurations To cable the controller to the disk shelves use the following guidelines m Connect each disk chain to two HBAs if available m Maximize the number of disk chains before adding disk shelves to an individual chain Balance the number of disk shelves across the disk chains in your system Disks with different capacities or rotation speeds cannot be mixed within a single disk shelf m To use mixed disk shelf types the controller must have only 4X4 port SAS 2 HBAs Do not use mixed disk shelves in the same chain each must be on their own SAS fabric The following list provides links to cable diagrams for standalone and clustered controllers m Oracle DE2 24 Disk Shelf Connecting DE2 Disk Shelves to the ZS3 2 7120 7320 ZS3 2 7320 Clustered Connecting DE2 Disk Shelves to the ZS3 4 7420 ZS3 4 7420 Clustered Sun Disk Shelf Connecting Sun Disk Shelves to the 7120 7320 7320 Clustered Connecting Sun Disk Shelves to the 7420 7420 Clustered Mixed DE2 24 and Sun Disk Shelves Connecting mixed DE2 24 and Sun Disk Shelves to the ZS3 2 7120 7320 ZS3 2 7320 Clustered Connecting mixed DE2 24 and Sun Disk Shelves to the ZS3 4 7420 ZS3 4 7420 Clus
10. Figure Legend 4 PCIe3 9 Rear Panel System Status LEDs 14 HD15 Video port 5 PCle 1 10 Serial Management port The serial management connector SER MGT is an RJ 45 port and provides a terminal connection to the SP console did The network management connector NET MGT is an RJ 45 port and provides an alternate terminal interface to the SP console E NET MoT There are four RJ 45 Gigabit Ethernet ports NET0 NET1 NET2 NET3 located on the motherboard that operate at 10 100 1000 Mbit sec These network interfaces must be configured before use Electrical Specifications The following list shows the electrical specifications for the 7120 Note that the power dissipation numbers listed are the maximum rated power numbers for the power supply The numbers are not a rating of the actual power consumption of the appliance Connectors Two C13 connectors which work on 110 220v outlets Input Nominal frequencies 50 60Hz Nominal voltage range 100 120 200 240 VAC m Maximum current AC RMS 13 8A 100 VAC AC operating range 90 264 VAC Output 3 3 VDCSTBY 3 0A Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7120 m 12 VDC 86 7A Power dissipation Max power consumption 1235 3 W Max heat output 4212 BTU hr Volt Ampere rating 1261 VA 240 VAC 0 98P F Internal Components The chassis has the following boards installed Note Field replaceable units FRU should onl
11. Position the replacement USB drive to where it installs onto the motherboard The USB drive installs into the USB socket that is most forward The socket is labeled USB 1 7 Gently insert the USB drive into the USB socket 8 Ifyou removed Riser 2 to access the USB drive reinstall Riser 2 9 Install the top cover 10 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 11 Connect the power cords to the power supplies 12 Usea pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit Battery Removing the battery is a cold service operation You must power offthe appliance before you remove the battery There will be a loss ofaccess to the storage unless the system is in a clustered configuration You might need to replace the battery if the storage controller fails to maintain the proper time when powered off and disconnected from the network You will need a small No 1 flat blade non metallic screwdriver or equivalent Power off the storage controller Extend the storage controller from the rack Remove the top cover and air filter Remove Riser 1 Ou ce coss B3 ES Press the tab ofthe battery receptacle forward to release the battery The battery pops up Chapter2 Hardware M
12. and clients then create directories or modify shares in any way those changes still exist after the rollback as you would expect Ifthe appliance is running version V and has previous rollback targets V 1 and V 2 and you revert all the way to version V 2 thereby skipping V 1 then you not only are removing the system software settings and system software for V but also for V 1 That is after a rollback to V 2 it is as if updates V 1 and V never happened However the software Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates upload images for V 1 and V will still be saved on the system and you can apply them again after the rollback if you wish by re executing the update If after applying an update the system is back up and running you can use either the BUI or the CLI to initiate a rollback to one oftwo previously applied updates If the system is not able to run at all after an update then use the fail safe rollback procedure Fail safe Rollback Administrators can execute a fail safe rollback of the system software from the serial console by selecting one of the other boot menu entries if present Although rollback can also be requested from the BUI or CLI rollback is offered from the boot menu because it is possible that rollback will be needed in scenarios where the new system software has completely failed i e has failed to even boot To rollback from the console access the se
13. clownfish maintenance system updates show Updates UPDATE DATE STATUS ak nas 2011 04 24 3 0 1 2 19 11 2 2012 6 24 17 14 19 current ak nas 2011 04 24 4 2 1 1 28 2012 11 5 03 11 34 waiting ak nas 2013 06 05 0 0 1 1 2 2013 6 19 12 58 18 unavailable Waiting media is listed as unavailable if the currently active software is not sufficiently recent to be able to upgrade to that waiting media In this example media has been downloaded for both 2011 1 4 2 and 2013 1 but the 2013 1 upgrade is unavailable because you first need to upgrade to at least 2011 1 4 2 Deferred updates The following incompatible updates are available Applying these updates will enable new software features as described below but will prevent older versions of the software from accessing the underlying resources You should apply deferred updates once you have verified that the current software update is functioning and a rollback is not required Applying deferred updates in a cluster will also update any resources on the cluster peer 1 Support for the passthrough x aclinherit property for shares clownfish maintenance system updates apply Applying deferred updates will prevent rolling back to previous versions of software Are you sure Y N clownfish maintenance system updates apply Chapter3 System Maintenance 213 Passthrough x Passthrough x Passthrough x Deferred Update For filesystems ACLs are inherited according to the aclinherit
14. configured asa mirrored pair Up to four 1 6TB SAS 2 Readzilla SSDs can fill slots 2 through 5 in order Figure Legend 1 Locate white 2 Service action required amber 3 OK Activity green Rear Panel The following graphic shows the rear panel Base configuration HBAs are not depicted in this illustration Figure Legend 1 Power supply unit 0 status LEDs OK green Power Supply Fail amber AC OK green 8 Network NET 10 100 1000 ports NETO NET3 2 Power supply unit 0 AC inlet 9 USB 2 0 ports 3 Power supply unit 1 status LEDs OK green Power Supply Fail amber AC OK green 10 PCIe slots 5 9 4 Power supply unit 1 AC inlet 11 Network management NET MGT port 5 System status LEDs Power green Attention amber Locate white 12 Serial management SER MGT port 6 PCIe slots 0 4 13 DB 15 video connector Chapter2 Hardware Maintenance 43 Z53 4 Figure Legend 7 Cluster card slot 44 Electrical Specifications The following list shows the electrical specifications for the controller Note The power dissipation numbers listed are the maximum rated power numbers for the power supply The numbers are nota rating of the actual power consumption ofthe appliance Input Nominal frequencies 50 60Hz AC operating range 200 240 VAC Maximum current AC RMS 12A 200 VAC Power Dissipation Max power consumption 1800 W Max heat output 6143 BTU hr Volt Ampere rati
15. controller fan modules In addition they contain fan module status LEDs and transfer I2C data for the fan modules m DriveBackplane The six drive backplane is a FRU and includes the connectors for the drives as well as the interconnect for the I O board Power and Locator buttons and system component status LEDs Each drive has an LED indicator for Power Activity Fault and Locate Components The controller components are shown in the following figure and identified in the table Figure Legend 1 Motherboard 7 CPUs and heatsinks 2 Low profile PCIe cards 8 Memory risers 3 Power supplies 9 Fan board 4 Power supply backplane 10 Fan modules 5 Drive backplane 11 Boot drives and SSDs 6 System lithium battery CPU and Memory The 7420 controller supports the following configurations 114 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 Two four or eight SGB DDR3 DIMMs installed on each riser accommodating 128GB 256GB or 512GB of memory for 2 0Ghz CPUs Four or eight 8GB DDR3 DIMMs installed on each riser accommodating 256GB or 512GB of memory for 2 0GHz and 2 4Ghz CPUs Four or eight 16GB DDR3 DIMMs installed on each riser accommodating 512GB or 1TB of memory for 2 4Ghz CPUs Note For proper cooling memory risers corresponding to unpopulated CPU sockets must have fillers installed Previous 7420 controller configurations support two or f
16. controller only Fan Power supply PSU Service processor SP controller only Clicking on a component type will display a list of all physical locations within the chassis where components may be present Clicking on a component within the list will highlight it within the appropriate chassis image Clicking on the information icon while over a row or double clicking a row will bring up a dialog with detailed information about the component The information displayed in the list depends on the component type but is a subset ofthe information available in the component detail Disks and service processors support additional operations described below Each component can report any or all ofthe following properties Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Hardware Property Description Label Human readable identifier for this component within the chassis This is typically but not necessarily equivalent to the label printed on the physical chassis FMRI Fault managed resource identifier FMRI for the component This is an internal identifier used to identify the component within faults and is intended for service personnel Active Problems For a faulted component links to active problems affecting the component Manufacturer Component manufacturer Model Component model Build Manufacturing build identifier This is used to identif
17. dory maintenance system updates download uncommitted set url ftp foo update pkg gz url ftp foo update pkg gz dory maintenance system updates download uncommitted gt set user bmc user bmc dory maintenance system updates download uncommitted set password Enter password password ORR RRR ek dory maintenance system updates download uncommitted gt commit Transferred 157M of 484M 32 3 Unpacking and Verifying Media After the file has been transferred it will be automatically unpacked and verified dory maintenance system updates download uncommitted gt commit Transferred 484M of 484M 100 done Unpacking done dory maintenance system updates list UPDATE DATE STATUS ak nas 2009 10 14 1 0 nd 2009 10 14 08 45 AKUP_WAITING Beginning an Upgrade To begin an upgrade select the update that constitutes the upgrade During this process an update health check will be performed to verify the appliance is ready to update Chapter 3 System Maintenance 211 Updates NOTE If the update is no longer appropriate for the system because you have skipped past its version number an error message may be provided From this context you can set any properties specific to the update including applying deferred updates For more information on the set of properties available for the particular update run the help properties command User controllable properties will begin with the update prefix c
18. the amber Service Required indicator on the HDD or SSD should be illuminated 2 Ifyouare not physically at the system turn on the locator indicator by clicking the locator icon 3 Pushthe release button on the HDD or SSD to open the latch 4 Grasp the latch 2 and pull the drive out of the drive slot 5 After 15 seconds navigate to the Hardware Maintenance screen and click the details icon on the system controller to verify that the software has detected that the drive is not present 6 Slide the replacement drive into the slot until it is fully seated 7 Closethelatch tolock the drive in place Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 The Sun ZFS Storage system software automatically detects and configures the new drive The device appears in the BUI Maintenance gt Hardware screen when you view details for the controller or drive shelf Fan Module Fan modules are hot swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities Caution Operating a controller for an extended period of time with fans removed reduces the effectiveness of the cooling system For this reason the replacement fan should be unpacked in advance and ready to insert into the controller chassis as soon as the faulted fan is removed 7120 or 7320 The fan modules and status indicators are hidden under a fan do
19. 3 5 SAS 2 drives The SSDs provide accelerated write operations and the high capacity HDDs provide reliable storage This disk shelf features dual redundant SAS Interface Module SIM boards and dual power supply with fan modules SAS 2 The SAS 2 Serial Attached SCSI 2 0 storage fabric supports a greater number of targets greater bandwidth higher reliability and bigger scale The scale and reliability improvements are achieved with SAS 2 disks you can daisy chain to as many as 36 shelves for certain systems for a total of 864 disks In addition the high performance SAS 2 HBA is designed for the Sun ZFS Storage 7000 series with a standard chip set to support a high density oftarget devices capable of attachment to 1024 targets With this fabric you are encouraged to apply entire shelves to pools so you can gain the benefits of No Single Point of Failure configurations and striping across the maximum possible number of devices The following shelves implement SAS 2 disks Oracle Storage Drive Enclosure DE2 24P Oracle Storage Drive Enclosure DE2 24C Sun Disk Shelf Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Front Panel The front panel consists of the drives and indicator lights Drive Locations The following figures show the location of the drives Oracle Storage Drive Enclosure DE2 24P aj looeeoocooosales tt tta e ics ss ls ls a s HF sie xi xix I sii xi
20. CONFIG DATE SYSTEM VERSION bfa614d7 1db5 655b cba5 bdObbOalefc4 2009 8 5 17 14 28 host 2009 08 04 1 0 cb2f005f cf2b 608f 90db fc7a0503db2a 2009 8 24 17 56 53 host 2009 08 18 1 0 Create a Configuration Backup The backup command saves a configuration backup You will be prompted to enter a descriptive comment for the backup and then enter done to execute the backup operation host maintenance system configs backup Backup Configuration Enter a descriptive comment for this configuration and click Commit to backup current appliance settings host maintenance system configs conf backup stepQ gt set commentz pre upgrade comment pre upgrade host maintenance system configs conf backup stepQ gt done host maintenance system configs Restore from a Saved Configuration The restore command reverts the system to a saved configuration You will be prompted to enter the universal unique identifier for the backup see the output of List above and then enter done to execute the restore Review the Restore Impact guidance above and confirm that it is ok to proceed host maintenance system configs restore Restore Select the configuration to restore host maintenance system configs conf restore stepQ gt set uuid 36756f96 b204 4911 8ed5 fefaf89cad6a uuid 36756f96 b204 4911 8ed5 fefaf89cad6a host maintenance system configs conf restore stepQ gt done Note Storage pools are not automatically unconfigured when you execute the restore c
21. CRU Part Number CPU Intel E5 2658 2 1G 8 core FRU 7019701 Memory DIMM CRU 7041603 16GB DDR3 1600 2Rx4 1 35V NIC HBA Options The following table describes the NIC HBA options for the ZS3 2 controller See PCIe Options for slot allocations Mktg Part Number Description SG SAS6 INT Z 8 port 6Gb s SAS 2 internal HBA SG XPCIE2FC QF8 Z 2 port 8Gb FC HBA 38 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 Mktg Part Number Description 7103791 4 port 4x4 SAS 2 6Gb s external HBA X1109A Z 2 port LOGbE SFP NIC X4242A 2 port Infiniband CX2 HCA 7101674 2 port 16Gb FC FCoE Universal HBA 7100477 4 port 1Gb Copper Ethernet UTP 7100488 2 port 10Gb Ethernet Copper Base T X2129A XCVR 850NM 1 10GPS SFP short reach X5562A Z 10GbE 1GbE SFP Transceiver long reach PCle Riser Configuration The three risers are labeled Riser 1 Riser 2 and Riser 3 Though similar the risers are not interchangeable Riser 1 installs at the left rear of the chassis Riser 2 at the center rear and Riser 3 at the right rear of the chassis Each riser can accommodate two PCIe cards Riser 1 contains slots 1 and 4 Riser 2 contains slots 2 and 5 Riser 3 contains slots 3 and 6 PCle Options The following table describes the supported PCle configuration options for the ZS3 2 controller Slots 1 and 2 are reserved for internal and external HBAs as shown in the following table
22. Checks System level health checks are provided to help ensure that no pathologies will interfere with the software update If a problem is encountered it is noted in the Alert Log and the update process is aborted System software updates will not proceed until all problems have been corrected You can manually run the same health checks in advance of any planned update This allows you to check the state of the system prior to scheduling an update maintenance window so you can correct any problems that could interfere with the update process Any problem report that is issued by a manual health check is identical to that issued by the health checks integrated in the update process As with the integrated health checks you are presented with a link to the Alert Log when problems are found If no problems are found the System Ready state transitions to Yes to indicate that the system is ready for software updates Note Running an update health check does not replace meeting required preconditions Precondition checks must also be executed and problems resolved prior to updating the system software BUI After you select and start an update update health checks may be issued from the software update dialog box in the BUI System ready Unchecked The system remains in the Unchecked state until the Check button is clicked During the health check operation an indicator shows its progress Oracle ZFS Storage Appliance Customer Service
23. Gouvernement des Etats Unis ou toute entit qui d livre la licence de ce logiciel ou l utilise pour le compte du Gouvernement des Etats Unis la notice suivante s applique U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use duplication disclosure modification and adaptation of the programs including any operating system integrated software any programs installed on the hardware and or documentation shall be subject to license terms and license restrictions applicable to the programs No other rights are granted to the U S Government Ce logiciel ou mat riel a t d velopp pour un usage g n ral dans le cadre d applications de gestion des informations Ce logiciel ou mat riel n est pas concu ni nest destin tre utilis dans des applications risque notamment dans des applications pouvant causer des dommages corporels Si vous utilisez ce logiciel ou mat riel dans le cadre d applications dangereuses il est de votre responsabilit de prendre toutes les mesures de secours de sauvegarde de redondance et autres mesures n cessaires son utilisation dans des conditions optimales de s curit Oracle Corporation et ses affili s d c
24. Portactive component is ok green Hardware m8 Portinactive component is not present grey vy Hardware component is faulted amber System Overview The main hardware page lists the system chassis a summary of its contents and any attached disk shelves on supported systems This provides an overview of the hardware present on the system The power icon iU located in the upper left of the view presents a dialog box to either power off reboot power cycle or reboot the appliance with diagnostics Only select the diagnostic reboot option when instructed by Oracle Service personnel because it could take a long time to complete and could have adverse results if not performed properly Do not perform a diagnostic reboot when system affecting operations are occurring such as upgrading firmware executing commands and configuring or unconfiguring storage The diagnostic reboot option is not available when using the system chassis power icon described below System Chassis The primary system chassis is shown on the top half of the view At the top left click the right arrow icon amp to get more detail about the chassis The indicator notes if there are any faulted components within the chassis and the name of the chassis The chassis name is initially set to the appliance name during installation To change the chassis name use the entry field on the Configuration gt Services gt System Identity screen
25. Removing fan modules memory PCIe cards DIMMs and USB are cold service operations You must shut off power from the appliance before you remove any of these components Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one ofthe following methods Log in to the BUI and click the power icon on the left side of the masthead SSH into the storage system and issue the maintenance system poweroff command SSH or serial console into the service processor and issue the stop SYS command Use a pen or non conducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown wherein all applications and files will be closed abruptly without saving press and hold the power button for at least four seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller isin standby power mode Extend the Storage Controller From the Rack Disconnect the AC power cords from the rear panel of the storage controller Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches Chapter 2 Hardware Maintenance 53 253 2 4 While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch
26. To remove the top cover Unlatch the fan module door pull the two release tabs back rotate the fan door to the open position and hold it there Chapter 2 Hardware Maintenance 139 7x20 Press the top cover release button and slide the top cover to the rear about a half inch 1 3 cm Lift up and remove the top cover 6 7420 To remove the top cover Simultaneously lift both cover latches in an upward motion 7 Lift up and remove the top cover 8 Using a small non metallic screwdriver press the latch and remove the battery from the motherboard The 7420 battery is shown here 140 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 10 11 The following figure shows the 7120 battery Press the replacement battery into the motherboard with the positive side facing upward 7120 or 7320 Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 To install the top cover Place the top cover on the chassis so that it hangs over the rear ofthe storage controller by aboutan inch 2 5 cm then slide the top cover forward until it seats Close the fan cover and engage the fan cover latches The cover must be completely closed for the storage controller to power on 7420 To install the top cover Place the top cover on the chassis 1 so that it is forward ofthe r
27. When adding PCIe cards populate from higher order slots 6 first toward lower order slots Note Any empty PCIe slots must have a filler panel installed Slot Mfg Part Number Description Max Note 1 7047852 8 port 6Gb s SAS 2 internal 1 Base configuration HBA 2 7067091 4 port 4x4 SAS 2 6Gb s 2 Base configuration external HBA 3 7067091 4 port 4x4 SAS 2 6Gb s 2 Second 4x4 port SAS 2 external HBA external HBA 3 7070195 4 port IGbE copper NIC 3 Optional recommended front end UTP 3 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup Chapter 2 Hardware Maintenance 39 253 2 Slot Mfg Part Number Description Max Note 3 7023303 2 port 16Gb universal 2 Optional FC target or initiator backup FC FCoE HBA 3 375 3696 01 2 port InfiniBand CX2 HCA 2 Optional recommended front end 3 7051223 2 port 10GbE optical NIC 2 Optional recommended front end 3 7070006 2 port 1OGbE copper NIC 2 Optional recommended front end 4 6 7070195 4 port IGbE copper NIC 3 Optional recommended front end UTP 4 6 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 4 6 7023303 2 port 16Gb universal 2 Optional FC target or initiator backup FC FCoE HBA 4 6 375 3696 01 2 portInfiniBand CX2 HCA 2 Optional recommended front end 4 6 7051223 2 port 10GbE optical NIC 2 Optional recommended front end 4 6 7070006 2 port IOGbE copper NIC 2 Optional recommended front end Optional C
28. ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Recursive Snapshots Slim ZIL Introduction This deferred update changes the layout of ZFS intentlog blocks to improve synchronous write performance These improvements rely on a ZFS pool upgrade provided by this update Before this update has been applied log records will continue to be written in the old format and performance may be reduced Applying this update is equivalent to upgrading the on disk ZFS pool to version 23 Snapshot Deletion Snapshot Deletion Deferred Update This deferred update increases snapshot deletion parallelism and reduces the size of transaction groups associated with snapshot deletion to improve systemic responsiveness These improvements rely on a ZFS pool upgrade provided by this update Before this update has been applied new snapshot data will be stored in the old format and deleted using the old algorithm Note that any snapshots created before this update is applied will also be deleted using the old algorithm Applying this update is equivalent to upgrading the on disk ZFS pool version 26 Recursive Snapshots Recursive Snapshots Deferred Update This deferred update allows recursive snapshots to be taken without suspending the ZFS intent log which greatly improves snapshot creation performance especially on heavily loaded controllers These improvements rely on a ZFS pool upgrade provided by this update Before this upda
29. access there are no copies of directory service password hashes for directory users stored in the configuration backup Only the user name user ID preferences and authorization settings for directory users are stored in the backup and then restored Following a configuration restore the local root administrative user password is not modified to the root password at the time ofthe backup The root password is left as is unmodified by the restore process to ensure that the password used by the administrator who is executing the restore process and thus has logged in using that password is retained If the administrator s intent was to also change the root password at the time of configuration restore that step must be executed manually following the restore using the normal administrative password change procedure Chapter3 System Maintenance 221 ConfigurationBackup 222 Managing Configuration Backups Using the BUI The following section outlines how various Configuration Backup tasks can be accomplished using the Configuration Backup area near the bottom of the Maintenance gt System screen in the BUI Create a Configuration Backup To create a backup simply click the Backup button above the list of saved configurations and follow the instructions You will be prompted to enter a descriptive comment for the backup Restore from a Saved Configuration Click the rollback icon on any saved configuration to begin th
30. all disks in the system Disk Shelves A list of disk shelves if supported is displayed at the bottom ofthe view The thumbnail to the left represents the front ofthe currently selected disk shelf Clicking on the right pointing arrow or double clicking on a row within the list will provide complete details about the disk shelf The state indicator will be orange if the chassis contains any faulted components The following fields are displayed in the list Property Description Name Name ofthe disk shelf used in faults and alerts This is initially set to the serial number ofthe disk shelf but can be changed by clicking on the name within the list Manufacturer Disk Shelf Manufacturer Model Disk Shelf Model Data Total size of all data disks within the disk shelf Cache Total size of all read optimized cache devices Readzillas within the drive shelf There are currently no supported disk shelves with read cache devices but this may not always be the case If there are no cache devices within the shelf then is displayed Chapter 1 Introduction 17 Hardware 18 Property Description Log Total size of all write optimized cache devices Logzillas within the drive shelf If there are no log devices within the shelf then is displayed Paths Total number of I O paths to the disk shelf The only supported configurations are those with multiple paths to all disks so this should read 2 u
31. and out id i 124 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 Install the replacement fan module into the storage controller fan slot Note The fan must bereplaced within one minute to avoid controller shutdown Apply firm pressure to fully seat the fan module Verify that the Fan OK status indicator is lit and that the fault status indicator on the replaced fan module is dim Close the top cover door immediately after replacing the fan to maintain airflow in the storage controller 6 7420 To replace the fan module Identify the faulted fan module by locating the corresponding Service Required status indicator or by clicking the X locate icon on Maintenance gt Hardware screen of the BUI for the fan you want to replace Lift the latch at the top of the fan module to unlock the fan module and pull the fan module out Unlock and insert the 7420 fan module Apply firm pressure to fully seat the fan module Chapter 2 Hardware Maintenance 125 7x20 Verify that the Fan OK status indicator is lit and that the fault status indicator on the replaced fan module is dim 7 Verify that the Top Fan status indicator the Service Required status indicators and the Locator status indicator Locator button are dim 8 Push the release tabs on the side of each rail and slowly slide the storage controller into the rack Power Supply Storage controllers are equipped with
32. as soon as the faulted fan is removed Chapter 2 Hardware Maintenance 75 Z53 4 The following illustration shows the fan modules in the ZS3 4 storage controller 1 Tolocate the chassis you want to service click the associated 3 locate icon on the Maintenance Hardware screen of the BUI or issue the set SYS LOCATE status on command at the service processor SP prompt Thelocate LED will flash on the controller chassis 2 Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack 3 From the front of the storage controller release the two slide release latches 4 While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch 5 Identify the faulted fan module by locating the corresponding Service Required status indicator or by clicking the 3 locate icon on Maintenance gt Hardware screen of the BUI for the fan you want to replace 6 Lift the latch at the top of the fan module to unlock the fan module and pull the fan module out 7 Unlock and insert the fan module 8 Apply firm pressure to fully seat the fan module 76 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 9 Verify that the Fan OK status indicator is lit and that the fault status indicator on the replaced fan module is dim 10 Verify that the Top Fan status indicator the Service Required status
33. availability serial attached SCSI SAS devices that provide expanded storage The main components are hot swappable including drives I O Modules IOMs or SAS Interface Module SIM boards for connecting to controllers and other disk shelves and dual load sharing power supply with fan modules This provides a fault tolerant environment with no single point of failure Component status is indicated with lights on the disk shelf and in the Maintenance Hardware screen ofthe BUI Refer to Disk Shelf Tasks for procedural information about replacing disk shelf components Oracle Storage Drive Enclosure DE2 24P tas tar LR CD RD CaO LO ER DD LR The Oracle Storage Drive Enclosure DE2 24P is a 2U chassis that supports 24 2 5 SAS 2 drives The high performance HDDs provide reliable storage and the SSDs provide accelerated write operations This disk shelffeatures dual redundant I O Modules IOMs and dual power supply with fan modules Oracle Storage Drive Enclosure DE2 24C Chapter2 Hardware Maintenance 143 Shelf 144 The Oracle Storage Drive Enclosure DE2 24C is a 4U chassis that supports 24 3 5 SAS 2 drives The SSDs provide accelerated write operations and the high capacity HDDs provide reliable storage This disk shelf features dual redundant I O Modules IOMs and dual power supply with fan modules Sun Disk Shelf 24x3 5 SAS 2 zu ZEE jE zm E zm The Sun Disk Shelf is a AU chassis that supports 24
34. be noted with a reason The following are example audit log entries as they would appear in the BUT Time User Host Summary Session Annotation 2009 10 12 05 20 24 root deimos Disabled ftp service 2009 10 12 03 17 05 root deimos User logged in 2009 10 11 22 38 56 root deimos Browser session timed out 2009 10 11 21 13 35 root lt console gt Enabled ftp service Chapter 3 System Maintenance 227 Logs 228 Phone Home If Phone Home is used this log will show communication events with Oracle support The following are example phone home entry as it would appear in the BUI Time Description Result 2009 10 12 Uploaded file cores ak 45e5ddd1 ce92 cl6e b5eb 9cb2a809 1 flc tar gz to Oracle OK 05 24 09 support BUI Use the Maintenance gt Logs screen to navigate logs using list controls and switch between logs using the local navigation buttons CLI Logs can be viewed under the maintenance logs section of the CLI Listing logs Use the show command to list available logs and the timestamp of the last log entry caji gt maintenance logs caji maintenance logs gt show Logs LOG ENTRIES LAST alert 2 2009 10 16 02 44 04 audit 42 2009 10 16 18 19 53 fltlog 2 2009 10 16 02 44 04 scrk 0 system 100 2009 10 16 03 51 01 Up to 100 recent entries for each log are visible using the CLI Viewing a log Logs may be selected for viewing with the show command caji maintenance logs gt
35. dataor Disk or log device that is 3 4 log disk label in disk shelf name has justone path configured in a pool does not or zero paths have two paths PAN1 Slot slot has a Revision B3 SAS HBA Revision CO A revision B3 SAS HBA is 4 or later required present PAN2 J4400 and J4500 disk shelves are not supported in this release Disk shelf shelves are present that are not supported for this software release V1 product is not supported in this release Controller product is not supported for this software release Take the following steps in the order listed above to resolve the issue detected during the upgrade health check N e re seat SIM and wait for steady Power LED before reconnecting cables Ifa SAS port LED is unlit check all connections and replace cables as needed Identify affected chassis then disconnect and remove faulted SIM After two minutes 3 Identify affected chassis and remove faulted disk After 30 seconds re seat disk and wait for steady or flashing LED 4 Contact Oracle Support for component service or replacement Deferred Updates Each update may come with new firmware or updates to external resources In general these updates are backwards compatible and applied automatically without user intervention There are exceptions however for non reversible updates These updates involve updating a resource external to the system software in a way that
36. did not exist at the time ofthe backup When restoring across software versions administrators should manually verify settings for new features following the restore Password maintenance The root password is not changed or reverted to the password at the time of the backup if it was different The current root password is maintained on the system across the restore For more details about passwords refer to the summary of Security Considerations Security Considerations A configuration backup contains information that is normally only accessible to the root administrative user on the appliance Therefore any configuration backup that is exported to another system or into a filesystem share must apply security restrictions to the backup file to ensure that unauthorized users cannot read the backup file Local user passwords are stored in the backup file in encrypted hashed format not as clear text However on the system access to these password hashes is restricted as they could be used as input to dictionary attacks Therefore administrators must carefully protect configuration backups that are exported either by restricting file access to the backup or by applying an additional layer of encryption to the entire backup file or both Directory user passwords are not stored in the appliance and therefore are not stored in the configuration backup If you have deployed a directory service such as LDAP or AD for administrative user
37. do so using the old much slower recursive rename code Applying this update is equivalent to upgrading the on disk ZFS pool to version 31 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ConfigurationBackup Multiple Initiator Groups per LUN Introduction This deferred update allows a LUN to be associated with more than one initiator group ata time If replicating LUNs with this update applied also apply the update to the replication target system s ConfigurationBackup Configuration Backup The configuration backup function enables the administrator to Backup the appliance configuration consisting of system metadata only such as the network configuration local users and roles service settings and other appliance metadata m Restore a previously saved configuration from a backup Export a saved configuration as a plain file so that it may be stored on an external server or included in a backup ofa share on the appliance itself Import a saved configuration that was previously exported from this system or another system making it available for a restore operation Backup Contents A configuration backup does include Metadata associated with the system as a whole such as settings for NTP NIS LDAP and other services Network device datalink and interface configuration User accounts roles and privileges preferences and encrypted passwords for local users
38. end 5 7051223 2 port optical 10GE NIC 6 Optional recommended front end 5 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup 5 375 3696 01 CX2 Infiniband HCA 4 Optional recommended front end 6 F375 3609 03 2 port SAS external HBA 6 Additional optional back end 7059331 for EU countries 6 7054739 4 port copper NIC 6 Optional recommended front end 6 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup 6 375 3606 03 CX Infiniband HCA 4 Optional recommended front end 6 7051223 2 port optical 10GE NIC 6 Optional recommended front end 118 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 Slot Mfg Part Number Description Max Note 6 375 3696 01 CX2 Infiniband HCA 4 Optional recommended front end 7 F375 3609 03 DP SAS external HBA 6 Additional optional back end 7059331 for EU countries 7 7054739 4 port copper NIC 6 Optional recommended front end 7 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup 7 375 3606 03 CX1 Infiniband HCA 4 Optional recommended front end 7 7051223 2 port optical 10GE NIC 6 Optional recommended front end 7 375 3696 01 CX2 Infiniband HCA 4 Optional recommended front end 8 F375 3609 03 2 port SAS external HBA 6 Base configuration 2 minimum 7059331 for EU countries 9 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup 9 7051223 2 port optical 1OGbE NIC 6 Optional recommend
39. especially along the connector edges Required Tools and Information The following tools are needed to service the CRUs Antistatic wrist strap Wear an antistatic wrist strap and use an antistatic mat when handling components such as HDDs or PCI cards When servicing or removing storage controller components attach an antistatic strap to your wrist and then to a metal area on the chassis Following this practice equalizes the electrical potentials between you and the storage controller Antistatic mat Place static sensitive components on an antistatic mat No 2 Phillips screwdriver Nonconducting No 1 flat blade screwdriver or equivalent Nonconducting stylus or pencil to power on the storage controller Chassis Serial Number To obtain support for your storage controller or to order new parts you need your chassis serial number You can find a chassis serial number label on the storage controller front panel on the left side Another label is on the top ofthe storage controller Alternatively click the Sun logo in the BUI masthead to obtain the serial number or issue the following command hostname maintenance hardware show Chapter2 Hardware Maintenance 121 7x20 122 Controller Replacement Tasks 7x20 Controller Replacement Tasks HDD or SSD l Identify the failed HDD or SSD by going to the Maintenance Hardware section ofthe BUI and clicking the drive details icon amp If you are physically at the system
40. hardware issues Lights out Management of each system for remote power control and console access Browser User Interface BUI The browser user interface The BUl is the graphical tool for administration of the appliance The BUI provides an intuitive environment for administration tasks visualizing concepts and analyzing performance data Chapter 1 Introduction 13 Hardware Hardware 14 The management software is designed to be fully featured and functional on a variety of web browsers Direct your browser to the system using either the IP address or host name you assigned to the NET 0 port during initial configuration as follows https ipaddress 215 or https hostname 215 The login screen appears The online help linked in the top right of the BUI is context sensitive For every top level and second level screen in the BUI the associated help page appears when you click the Help button Command Line Interface CLI The CLI is designed to mirror the capabilities of the BUI while also providing a powerful scripting environment for performing repetitive tasks The following sections describe details of the CLI When navigating through the CLI there are two principles to be aware of Tab completion is used extensively if you are not sure what to type in any given context pressing the Tab key will provide you with possible options Throughout the documentation pressing Tab is presented as the word tab in
41. is equivalent to upgrading the on disk ZFS pool to version 21 Replication Replication Deferred Update The 2010 Q1 release stores replication configuration differently than 2009 Q3 and earlier releases This update migrates existing target action and replica configuration created under an earlier release to the new form used by 2010 01 and later After upgrading to 2010 01 but before this update is applied incoming replication updates for existing replicas will fail Replicas received under earlier releases will not be manageable via the BUI or CLI though they will occupy space in the storage pool Additionally the system will not send replication updates for actions configured on earlier releases After applying this update incoming replication updates for replicas originally received on earlier releases will continue normally and without a full resync The system will also send incremental replication updates for actions configured under earlier releases Received Properties 216 Received Properties Deferred Update The 2010 01 feature that enables administrators to customize properties on replicated shares relies on a ZFS pool upgrade This upgrade provides support of persistent local changes to received properties Before this upgrade has been applied the system will not allow administrators to change properties on replicated shares Applying this update is equivalent to upgrading the on disk ZFS pool to version 22 Oracle
42. located on these boards Do not touch the components without using antistatic precautions especially along the connector edges Required Tools and Information The following tools are needed to service the CRUs Chapter 2 Hardware Maintenance 73 Z53 4 74 Antistatic wrist strap Wear an antistatic wrist strap and use an antistatic mat when handling components such as HDDs or PCI cards When servicing or removing storage controller components attach an antistatic strap to your wrist and then to a metal area on the chassis Following this practice equalizes the electrical potentials between you and the storage controller Antistatic mat Place static sensitive components on an antistatic mat No 2 Phillips screwdriver Nonconducting No 1 flat blade screwdriver or equivalent m Nonconducting stylus or pencil to power on the storage controller Chassis Serial Number To obtain support for your storage controller or to order new parts you need your chassis serial number You can find a chassis serial number label on the storage controller front panel on the left side Another label is on the top ofthe storage controller Alternatively click the Oracle logo in the BUI masthead to obtain the serial number or issue the following command hostname maintenance hardware show Controller Replacement Tasks ZS3 4 Controller Replacement Tasks HDD or SSD l Identify the failed HDD or SSD by going to the Maintenance
43. locked interface configuration is required to gather diagnostic information from a failed controller WARNING Failure to configure locked management interfaces on a cluster may lead to longer than necessary fault diagnosis and resolution times Chapter2 Hardware Maintenance 167 Cabling Cabling 168 Observing and Clearing CPU faults from ILOM Log in to the server as root using the ILOM CLI To view server faults type the following command to list all known faults on the system show SP faultmgmt The server lists all known faults for example SP faultmgmt Targets 0 SYS MB P0 Properties Commands cd show To clear the CPU fault type the following command set SYS MB Pn clear fault action true For example to clear a fault on CPO gt set SYS MB PO clear fault action true Are you sure you want to clear SYS MB P y n y See Also Cluster Configuration Connecting to Attached Storage The cabling information in this section shows a subset of the supported configurations for Oracle ZFS Storage controllers For the maximum number of disk shelves per controller see the Summary of Controller Configurations To cable the controller to the disk shelves use the following guidelines Connect each disk chain to two HBAs if available Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Cabling Cabling Maximize the number of disk chains befor
44. model Cable Diagrams 2320 qp qnl z SHELF fig 1 Standalone controller with one HBA and one disk shelf in a single chain fig 2 One HBA and two disk shelves in a single chain Chapter2 Hardware Maintenance 179 7320 Cluster Cabling fig 3 One HBA and six disk shelves in a single chain 7320 Cluster Cabling 180 Connecting the 7320 Clustered Controller to Sun Disk Shelves The following figures show a subset of the supported configurations for Oracle ZFS Storage 7320 clustered controllers with one HBA Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model Cable Diagrams fig 1Clustered controllers with one HBA and one disk shelf in a single chain Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 Cabling fig 3 One HBA and six disk shelves in a single chain 7420 Cabling Connecting the 7420 Standalone Controller to Sun Disk Shelves 2 and 6 HBAs The following figures show a subset of the supported configurations for Oracle ZFS Storage 7420 standalone controllers with two and six HBAs Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model Cable Diagrams Chapter2 Hardware Maintenance 181 7420 Cabling 182 E fig 1Stand
45. property on the filesystem or inherited from the project Previous versions of software allowed four options for this setting discard noallow restricted and passthrough The 2009 Q2 0 0 release introduces a new option passthrough x with slightly different semantics as described in the product documentation Same as passthrough except that the owner group and everyone ACL entries inherit the execute permission only ifthe file creation mode also requests the execute bit The passthrough mode is typically used to cause all data files to be created with an identical mode in a directory tree An administrator sets up ACL inheritance so that all files are created with a mode such as 0664 or 0666 This all works as expected for data files but you might want to optionally include the execute bit from the file creation mode into the inherited ACL One example is an output file that is generated from tools such as cc or gcc If the inherited ACL doesn t include the execute bit then the output executable from the compiler won t be executable until you use chmod 1 to change the file s permissions In order to use this new mode the storage pool must be upgraded If you choose not to upgrade the pool and attempt to use this new property you will get an error indicating that the storage pool needs to be upgraded first There is no other implication of applying this update and it can be ignored if there is no need to use th
46. removing non faulted drives label each drive with the number of the slot from which it was removed and reinstall each drive into the same slot Faulted drives are indicated by an amber LED Go to the Maintenance Hardware section of the BUI click the right arrow icon at the beginning of the appropriate disk shelf row and click the information icon for the appropriate drive to view details or click the locate icon 3 to turn on the locator LED Important Do not remove acomponent if you do not have an immediate replacement The disk shelf must not be operated without all components in place Locate the failed disk drive at the front of the chassis Press the release button or latch to release the drive lever Chapter 2 Hardware Maintenance 157 Shelf 3 Pullthe drive lever fully open to unlock and partially eject the drive from the chassis 4 Grasp the middle of the drive body and pull it toward you to remove it from the chassis 5 Ensure the new drive lever is in the fully extended position 158 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf 6 While constantly pushing toward the pivot point of the lever slide the drive fully into the chassis slot 7 Press the drive lever closed until it locks in place For vertically oriented drives push down on the top of the drive if it is higher than surrounding drives to properly seat it The Activity LED will be steady green t
47. service only 2 Power OK indicator 5 SAS 2 Port 2 8 For Oracle service only 3 SAS 2 Port 0 6 Host port activity indicators SIM Board Indicators The following disk shelves have SIM boards Sun Disk Shelf The following figure shows the SIM board indicators for the Sun Disk Shelf Chapter 2 Hardware Maintenance 151 Shelf Figure Legend 1 AC power indicator 6 Power switch 2 DC power indicator 7 Port fault indicator 3 Fan fault indicator 8 Port OK indicator 4 Power supply fault indicator 9 SIM board OK indicator green SIM board fault indicator amber 5 Universal power connector 10 SIM locator indicator Power Supply Indicators The following figure shows power supply with fan module indicators for these disk shelves Oracle Storage Drive Enclosure DE2 24P Oracle Storage Drive Enclosure DE2 24C 152 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Figure Legend 1DC power fail indicator 6 Universal power input connector 2 Fan fail indicator 7 Power cord tie wrap 3 AC power fail indicator 4 Power supply status indicator 5 Power on off switch The following figure shows power supply with fan module indicators for these disk shelves Sun Disk Shelf Chapter 2 Hardware Maintenance 153 Shelf 154 Figure Legend 1 Cooling fan status indicator 6 Power cord tie wrap 2 AC power status indicato
48. storage controller forward until the slide rails latch 7120 or 7320 To remove the top cover Unlatch the fan module door pull the two release tabs back rotate the fan door to the open position and hold it there Press the top cover release button and slide the top cover to the rear about a half inch 1 3 cm Lift up and remove the top cover 7420 To remove the top cover Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 Simultaneously lift both cover latches in an upward motion Lift up and remove the top cover 7 Locate the PCIe card position in the storage controller see Single and Cluster Controller Configurations for the 7320 the 7120 Overview or 7420 Overview 8 7120 or 7320 To replace the PCIe card Disconnect any data cables connected to the cards on the PCIe riser you want to replace Label the cables for proper connection later Loosen the two captive Phillips screws on the end of the rear panel crossbar and lift the crossbar up and back to remove it Loosen the captive retaining screw holding the front end ofthe riser and the Phillips screw on the end of the riser Chapter2 Hardware Maintenance 135 7x20 Lift the riser up to remove it from the storage controller filtered compressed air if necessary Seat the replacement PCIe card in the slot ofthe riser and connect the cables Align the riser together with any attached PCIe cards
49. the BUI Inacluster environment an ILOM connection should be made to each controller The server ILOM provides options for 1 network and ii serial port connectivity Network connection is the preferred choice as the ILOM serial port does not always allow adequate means of platform data collection WARNING Failure to configure ILOM connectivity may lead to longer than necessary hardware fault diagnosis and resolution times Management Port Configuration All standalone controllers should have at least one NIC port configured as a management interface Select the Allow Admin option in the BUI to enable BUI connections on port 215 and CLI connections on ssh port 22 All cluster installations should have at least one NIC port on each controller configured as a management interface as described above In addition the NIC instance number must be unique on each controller For example nodeA uses igb0 and nodeB uses igb1 so that neither may be used as a cluster data interface In addition these interfaces must be locked to the controller using the Configuration gt Cluster option in the BUL In some cases this may require installation ofan additional network interface card on each controller in a cluster configuration If access to the appliance data interfaces is impossible for any reason the management network interface will maintain BUI and CLI access During a cluster takeover interfaces are taken down on the failed controller So
50. the fan modules Storage Drive Backplane The storage drive backplane is a FRU and includes the connectors for the storage drives as well as the interconnect for the I O board power and locator buttons and system component status LEDs The system has a 12 disk backplane Each drive has an LED indicator for Power Activity Fault and Locate Chapter 2 Hardware Maintenance 93 7120 1 0 Components The following figure and legend show the I O components of the 7120 system Figure Legend 1 Top Cover 3 Hard Disk Drives 2 Right Control Panel Light Pipe Assembly 4 Left Control Panel Light Pipe Assembly Cables The following figure and legend show the storage controller internal cables 94 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7120 Note The rear boot drives are not depicted in this illustration Cable Connection 1 Storage Drive Data Cable Connection between the HBA PCI Express Card and the storage drive backplane Cable Connection 2Ribboncable Connection is between the power distribution board and the motherboard Chapter2 Hardware Maintenance 95 7120 96 CPU and Memory The 7120 motherboard has 18 slots in two groups that hold industry standard DDR3 DIMMs The standard memory configuration is 48GB 6x8GB DDR 1333 low voltage LV DIMMS Following are the replaceable CPU and memory components ofthe 7120 system Part
51. to Disk Shelf Procedures for replacing disk shelf components Prerequisites Read the ZS3 2 Overview section to become familiar with the replaceable parts of the system Follow the instructions in the Safety Information and Required Tools and Information sections Safety Information This section contains safety information that you must follow when servicing the storage system For your protection observe the following safety precautions when setting up your equipment Chapter 2 Hardware Maintenance 51 253 2 52 Do not remove the side panels or run the storage system with the side panels removed Hazardous voltage is present that could cause injury The covers and panels must be in place for proper air flow to prevent equipment damage Follow all cautions warnings and instructions marked on the equipment and described in Important Safety Information for Oracle s Hardware Systems included with your system Ensure that the voltage and frequency of your power source match the voltage inscribed on the electrical rating label Follow the electrostatic discharge safety practices Electrostatic discharge ESD sensitive devices such as PCI cards HDDs SSDs and memory cards require special handling Circuit boards and HDDs contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these
52. to Oracle You will receive an alert when the bundle has finished uploading Please save the following filename as Oracle support personnel will need it in order to access the bundle cores ak 9a4c3d7b 50c5 6eb9 c2a6 ec9808ae1cd8 tar gz As the message indicates you must provide this filename to support personnel in order for them to retrieve your bundle Manage bundles from the maintenance system bundles context in the CLI as follows loader maintenance system bundles loader maintenance system bundles list BUNDLE STATUS PROGRESS cores ak 9a4c3d7b 50c5 6eb9 c2a6 ec9808ae1cd8 tar gz Uploading 7 loader maintenance system bundles Bundles are identified by the filename omitting the ak prefix and the file type suffix To delete a support bundle use the destroy command To view details use the select and list commands loader maintenance system bundles select 9a4c3d7b 50c5 6eb9 c2a6 ec9808aelcd8 loader maintenance system bundles 9a4c3d7b 50c5 6eb9 c2a6 ec9808aelcd8 list Properties filename cores ak 9a4c3d7b 50c5 6eb9 c2a6 ec9808ael1cd8 tar gz status uploading step progress 14 709744730821669 These read only properties indicate that the appliance is 1496 ofthe way through uploading the file To retry a failed upload or cancel a pending operation enter the retry and cancel commandis respectively Initial Setup Initial setup will step through the tasks performed as part ofthe initial configuration This will n
53. when using static IP configuration IPv6 is not supported Subnet Dotted decimal subnet when using static IP configuration Default Gateway IPv4 default gateway address Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Hardware Changing multiple values in conflicting ways such as changing static IP assignments while in DHCP mode has undefined behavior CLI Hardware status details are available in the CLI under the maintenance hardware section Use the show command to list the status of all components The List command will list available chassis which can be selected and then viewed using show tarpon maintenance hardware show Chassis NAME STATE MANUFACTURER MODEL chassis 000 08390CJ01A ok Sun Microsystems Inc Sun Storage 7320 cpu 000 CPU 0 ok AMD Quad Core AMD Op cpu 001 CPU 1 ok AMD Quad Core AMD Op cpu 002 CPU 2 ok AMD Quad Core AMD Op cpu 003 CPU 3 ok AMD Quad Core AMD Op disk 000 HDD 0 ok STEC MACH8 IOPS disk 001 HDD 1 ok STEC MACH8 IOPS disk 002 HDD 2 absent disk 003 HDD 3 absent disk 004 HDD 4 absent disk 005 HDD 5 absent disk 006 HDD 6 ok HITACHI HTE5450SASUN500G disk 007 HDD 7 ok HITACHI HTE5450SASUN500G fan 000 FT 0 ok unknown ASY FAN BOARD H2 fan 001 FT 0 FM 0 ok Sun Microsystems Inc 541 2068 fan 002 FT 0 FM 1 ok Sun Microsystems Inc 541 2068 fan 003 FT 0 FM 2 ok Sun Microsystems Inc 541 2068 fan 004 FT 1 ok unknown ASY FAN BOARD H2 fan 0
54. 0 1xCPU 24GB 6 6TB 7014525 NIC HBA Options The following table describes NIC HBA options for the 7120 Mktg Part Number Description Mfg Part Number SG XPCIESAS GEN2 Z 2 port External Sun Thebe SAS x4 HBA PCIe 594 5889 01 SG XPCIE2FC QF8 Z 2 port FC HBA 8Gb PCIe 594 5684 01 X4446A Z 4 port PCI E Quad GigE UTP 594 4024 01 X4237A N 2 port 4X IB HCA PCle 594 5862 02 X1109A Z 2 port 10Gig SFP NIC PCle 594 6039 01 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7320 7320 PCle Options The following table describes the supported PCIe configuration option summary for the 7120 Slot Type SunPartNumber Vendor Part Number Description Note 0 PCIe 540 7975 03 Sun Aura Internal Flash HBA Logzilla Base Configuration OBSOLETE 0 PCIe 375 3481 01 Intel EXPI9404PT QP Copper NIC Optional Recommended Front end 0 PCIe 375 3617 01 Intel Niantic DP Optical 10GE NIC Optional Recommended Front end 0 PCIe 371 4325 01 QLogic 8Gb DP FC HBA Optional FC Target or Initiator Bac 0 PCIe 375 3606 01 Mellanox MHJH29 XTC InfiniBand HCA Optional Recommended Front end 1 PCIe 375 3617 01 Intel Niantic DP Optical 10GE NIC Optional Recommended Front end 1 PCIe 375 3606 01 Mellanox MHJH29 XTC InfiniBand HCA Optional Recommended Front end 1 PCIe 375 3481 01 Intel EXPI9404PT QP Copper NIC Optional Recommended Front end 1 PCIe 371 4325 01 QLogic 8Gb DP FC HBA Optional
55. 0 Ethernet ports The following table describes base configurations for the 7320 Mktg Part Number Description Mfg Part Number TA7320 24A 7320 2xCPU 24GB Single 597 1060 01 7104054 7320 2xCPU 96GB Single 7045900 TA7320 24A HA 7320 2xCPU 24GB Cluster 597 1061 01 7104055 7320 2xCPU 96GB Cluster 7045903 Following are the PCIe configuration options for a single controller All PCIe cards are low profile and must be fitted with low profile mounting brackets Slot Type Part Number Vendor Part Description Note 0 PCIe 375 3617 01 Intel Niantic DP Optical 1OGE NIC Optional Recommended Front end 0 PCIe 375 3696 01 Mellanox InfiniBand HCA Optional Recommended Front end 0 PCIe 375 3606 03 MHJH29 XTC InfiniBand HCA Optional Recommended Front end 0 PCIe 375 3481 01 Intel EXPI9404PT QP Copper NIC Optional Recommended Front end 0 PCIe 371 4325 01 QLogic 8Gb DP FC HBA Optional FC Target or Initiator Backup 1 PCIe 375 3617 01 Intel Niantic DP Optical IOGE NIC Optional Recommended Front end 1 PCIe 375 3696 01 Mellanox InfiniBand HCA Optional Recommended Front end 1 PCIe 375 3606 03 MHJH29 XTC InfiniBand HCA Optional Recommended Front end 1 PCIe 375 3481 01 Intel EXPI9404PT QP Copper NIC Optional Recommended Front end 1 PCIe 371 4325 01 QLogic 8Gb DP FC HBA Optional FC Target or Initiator Backup 2 PCIe 375 3609 03 Sun Thebe External SAS HBA Base Configuration 7320 Cluster Config
56. 05 FT 1 FM 0 ok Sun Microsystems Inc 541 2068 fan 006 FT 1 FM 1 ok Sun Microsystems Inc 541 2068 fan 007 FT 1 FM 2 ok Sun Microsystems Inc 541 2068 memory 000 DIMM 0 0 ok HYNIX 4096MB DDR II 66 DIMM 0 1 ok HYNIX 4096MB DDR II 66 memory 001 A 5th column for serial number SERIAL has been truncated in the above example as has the length of this list Component Properties Ifa particular component is selected detailed information about its properties are reported The following properties are supported with the corresponding BUI property name For a description ofa particular property see the description above Chapter 1 Introduction 21 Hardware 22 CLI Property BUI Property build Build cores Cores device N A faulted status indicator label Label locate writable status indicator manufacturer Manufacturer model Model offline status indicator writeable part Part present status indicator revision Revision serial Serial size Size speed Speed type combined with use use Type When viewing a disk that is active as a hot spare the detach command is also available Viewing CPU Details For example the following shows details for component CPU 0 tarpon maintenance tarpon maintenance tarpon maintenance chassis 000 tarpon maintenance chassis 000 Properties label present faulted manufacturer model part hardware
57. 091 4 port 4x4 SAS 2 6Gb s 4 Additional optional back end external HBA 7 371 4325 02 2 port 8Gb FC HBA 4 Optional recommended front end 7 375 3696 01 2 portInfiniband CX2 HCA 4 Optional recommended front end 7 7051223 2 port 10Gb Optical Ethernet 6 Optional recommended front end 7 7070195 4 port IGbE copper NIC 4 Optional recommended UTP front end 7 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 8 7067091 4 port 4x4 SAS 2 6Gb s 4 Base configuration external HBA Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 ZS3 2 Slot Mfg Part Number Description Max Note 9 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 9 7070006 2 port 10Gb copper Ethernet 4 Optional recommended front end 9 7070195 4 port 1GbE copper NIC 4 Optional recommended UTP front end 9 7051223 2 port IOGbE optical NIC 6 Optional recommended front end Attached Storage The ZS3 4 single and cluster controller configurations allow one to six chains of 1 to 6 disk shelves Any combination of disk only and Logzilla capable shelves may be combined within the chain in any order The cabling configurations are unchanged See Disk Shelf Overview for component specifications and diagrams ZS3 2 CRU Maintenance Procedures This section provides instructions on how to replace customer replaceable components CRUs in the Oracle ZFS Storage ZS3 2 controller Refer
58. 14 1 0 20091014T084500Z Creating system boot ak nas 2009 10 14 1 0 done Creating system root ak nas 2009 10 14 1 0 done As the upgrade proceeds the latest message will be printed You can cancel the upgrade at any time by pressing C at which point you will be prompted for confirmation Updating from ak nasQ2009 10 11 1 0 Backing up smf 5 done Loading media metadata C Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates This will cancel the current update Are you sure Y N y error interrupted by user dory maintenance system updates ak nasQ2009 10 14 1 0 nd Rolling Back To roll back to an earlier version select the update that corresponds to that version and execute the rollback command You will be asked to confirm that you wish to execute a rollback and then the system will reboot and execute the rollback Unlike fail safe rollback you will not be asked for further confirmation when the system reboots Removing Update Media To remove update media use the dest roy command specifying the update to be removed dory maintenance system updates destroy ak nas 2009 10 14 1 0 nd This will destroy the update ak nasQ2009 10 14 1 0 nd Are you sure Y N y dory maintenance system updates Applying Deferred Updates CLI To see if there are any available deferred updates run the show command If deferred updates are available you can use the apply command
59. 5 video connector 16 SSD 5 optional 5 Service Processor SP OK LED green 11 Boot drive 0 mirrored 6 Fan CPU Memory Service Required LED 12 Boot drive 1 mirrored 110 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 System Drives The 7420 M2 has two 900GB SAS 2 system boot drives in slots 0 and 1 configured as a mirrored pair Up to four 1 6TB SAS 2 Readzilla SSDs may fill slots 2 through 5 in order In a 7420 M2 cluster the number of solid state drives SSDs installed in each controller can vary 7420 controllers have two 500GB SATA system boot drives in slots 0 and 1 configured as a mirrored pair Zero two three or four 512GB SSDs may optionally fill slots 2 through 5 in order In a 7420 cluster the number of SSDs installed in both controllers must match 7420 M2 10K RPM 6 Gbps mz oz S8 uw lt i PE R Figure Legend 1 Locate white 2 Service action required amber 3 OK Activity green Rear Panel The following graphic shows the rear panel of the controller Base configuration HBAs are not depicted in this illustration Chapter 2 Hardware Maintenance 111 7420 Figure Legend 1 Power supply unit 0 status LEDs OK green Power Supply Fail amber AC OK green 8 Network NET 10 100 1000 ports NETO NET3 2 Power supply unit 0 AC inlet 9 USB 2 0 ports 3 Power supply unit 1 status LEDs OK green Power Supply Fai
60. 59331 for EU countries 2 7054739 4 port copper NIC 6 Optional recommended front end Chapter 2 Hardware Maintenance 117 7420 Slot Mfg Part Number Description Max Note 2 371 4325 02 8Gb DP FC HBA 6 Optional FC target or initiator Backup 2 375 3606 03 CX Infiniband HCA 4 Optional recommended front end 2 7051223 2 port optical 10GE NIC 6 Optional recommended front end 2 375 3696 01 CX2 Infiniband HCA 4 Optional recommended front end 3 F375 3609 03 2 port SAS external HBA 6 Additional optional back end 7059331 for EU countries 3 7054739 4 port copper NIC 6 Optional recommended front end 3 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup 3 375 3606 03 CX Infiniband HCA 4 Optional recommended front end 3 7051223 2 port optical 10GE NIC 6 Optional recommended front end 3 375 3696 01 CX2 Infiniband HCA 4 Optional recommended front end 4 7054739 4 port copper NIC 6 Optional recommended front end 4 375 3606 03 CX Infiniband HCA 4 Optional recommended front end 4 7051223 2 port optical 10GE NIC 6 Optional recommended front end 4 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup 4 375 3696 01 CX2 Infiniband HCA 4 Optional recommended front end C 7056175 Cluster Controller 200 1 Cluster base configuration 5 7054739 4 port copper NIC 6 Optional recommended front end 5 375 3606 03 CX Infiniband HCA 4 Optional recommended front
61. 66 With the ejector arms in the full open position align the new SIM board with the open slot and slide it into the tray until the ejector arms contact the tray connectors and begin to swing closed Swing both ejector arms in until they are flush with the SIM board panel to seat the board Tighten both captive screws to secure the board Reconnect the SAS interface cables to their original locations Wait approximately 60 seconds for the SIM board to complete its boot process at which time the Power LED should be solid green and the SIM locate LED should be off Goto the Maintenance Hardware section of the BUI click the right arrow icon Q atthe beginning of the appropriate disk shelf row and then click Slot to verify that the SIM board icon is green for the newly installed SIM board Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Faults Faults Hardware Faults This section describes connecting to the controller Service Processor SP and configuration considerations for maximum serviceability In rare cases faults associated with uncorrectable CPU errors are not diagnosable or displayed in the controller These faults will be preserved by and observable on the ILOM The following sections describe how to connect to and manage faults for these cases Connect to ILOM Connect to the server ILOM Service Processor on the server platform to diagnose hardware faults that do not appear in
62. Cable Diagrams fig 1 Standalone controller with one HBA and one disk shelf in a single chain Chapter2 Hardware Maintenance 171 DE2 to ZS3 2 7120 7320 Standalone fig 5 Four disk shelves in a single chain 172 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 DE2 to ZS3 2 7320 Clustered DE2 to ZS3 2 7320 Clustered Connecting ZS3 2 7320 Clustered Controllers to Disk Shelves The following figures show a subset of the supported configurations for Oracle ZFS Storage ZS3 2 7320 clustered controllers with one HBA Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model Cable Diagrams fig 2 One HBA and two disk shelves in two chains Chapter 2 Hardware Maintenance 173 DE2 to ZS3 4 7420 Standalone fig 4Four disk shelves in a single chain DE2 to ZS3 4 7420 Standalone Connecting the ZS3 4 7420 Standalone Controller to Disk Shelves 3 HBAs The following figures show a subset of the supported configurations for Oracle ZFS Storage ZS3 4 7420 standalone controllers with three HBAs Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model Cable Diagrams 174 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 DE2 to ZS3 4 7420 Standalone fig 4 Three
63. EN pes Prerequisttesaduetonttuotmimonteuuaehboteddaude uen Taco de DE SEA Safety Information on naan a sn ie iere X ORO e dues Electrostatic Discharge Precautions AET e ET E ariete eomm E edad onde seda unravel oti Et cl Eaa aaa AE AAT ER R Hardware Faults aacra tte rto RR esanian e Ug e nae Eve Sa dee Cet Min Gs sed sti tesd sass vetet otitetedenuiedte t dite min iode E ftindentaintes Connecting to Attached Storage Cabling E E d aree qt OR DEO E E E E ATO OR Unos Connecting to Attached Storage reete dine biete tue dli edited lotos Summary of Controller Configurations sissisota Maximum Controller Configurations eerte tentent tenente tentent DE2 to ZS3 2 7120 7320 Standalone Connecting ZS3 2 7120 7320 Standalone Controllers to Disk Shelves 171 DE210 753 2 7320 C steted 2 anosen E o Ame eee BRI Od ARE 173 Connecting ZS3 2 7320 Clustered Controllers to Disk Shelves ss 173 DE to Z53 4 7420 Standalone itt itr irt i Ht tede eigens 174 Connecting the ZS3 4 7420 Standalone Controller to Disk Shelves 3 HBAS 174 DE2to Z 3 4 7420 Clustered erret titt Epi tie t p be ie i Rn Re Ra hn Rh raga 176 Connecting the ZS3 4 7420 Clustered Controller to Disk Shelves 3 HBAS 176 7120 7320 Cabling scenes aae het dieto deni hina dedos dide an Onde eho a dive 179 Connecting 7120 7320 Standalone Controllers to S
64. FC Target or Initiator Bac 3 PCle 375 3665 01 Sun Thebe INT Internal SAS HBA Base Configuration 4 PCIe 375 3481 01 Intel EXPI9404PT QP Copper NIC Optional Recommended Front end 4 PCIe 371 4325 01 QLogic 8Gb DP FC HBA Optional FC Target or Initiator Bac 4 PCIe 375 3609 03 Sun Thebe EXT 8P 6Gb s SAS HBA Additional Optional Back end Attached Storage The 7120 standalone configurations allow a single chain of 1 or 2 disk shelves Write optimized Logzilla SSDs are not supported in the expansion storage for the 7120 The disk shelves must be fully populated with 24 HDDs Half populated shelf configurations are not supported 7320 Hardware Overview Use the information in this section to prepare to service replaceable components ofthe 7320 system After you have reviewed this section refer to these procedural instructions Controller Tasks replace storage controller components Disk Shelf Tasks replace disk shelf components Chapter2 Hardware Maintenance 99 7320 100 Chassis Overview The Sun ZFS Storage 7320 consists of either a single storage controller or two storage controllers in a high availability cluster configuration Both the single and clustered configuration supports one to six disk shelves The 7320 controller base configuration includes two CPUs built in 4 x 1Gb s front end GigE ports redundant power supplies NIC options for expanded front end support tape backup InfiniBand and dual port SAS HBA fo
65. FS filesystem shared using data protocols SIM SAS Interface Module Snapshot an image of a share SSD Solid State Drive SSH Secure Shell Statistic a metric visible from Analytics Storage Controller the head node of the appliance Support Bundle auto generated files containing system configuration information and core files for use by remote support in debugging system failures Title Bar local navigation and function section of BUI screen Updates software or firmware updates WebDAV Web based Distributed Authoring and Versioning ZFS on disk data storage subsystem 233 234
66. HBASAS 2 Version min 7105725 4x10 core 1TB 16GB Four 1 6TB Two 900GB 4X4 port 2013 1 0 2 40GHz DIMMs Chapter 2 Hardware Maintenance 41 Z53 4 42 Refer to the http www oracle com us products servers storage storage unified storage index html http www oracle com us products servers storage storage unified storage index htm1 for the most recent component specification Chassis Dimensions The ZS3 4 controller chassis fits in a standard equipment rack and occupies three rack units 3RU in height The chassis dimensions are as follows Dimension Measurement Dimension Measurement Height 13 3cm 5 25in Depth 70 6 cm 27 8 in Width 43 7 cm 17 19in Weight 16 36 kg 96 lbs Front Panel Figure Legend 1 Locator LED and button white 9 USB 2 0 Connectors 2 Service Required LED amber 10 DB 15 video connector 3 Power OK LED green 11 Boot drive 0 4 Power button 12 Boot drive 1 required 5 Service Processor SP OK LED green 13 Solid state drive 2 optional 6 Fan CPU Memory Service Required LED 14 Solid state drive 3 optional 7 Power Supply PS Service Required LED 15 Solid state drive 4 optional 8 Over Temperature Warning LED 16 Solid state drive 5 optional Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 System Drives The ZS3 4 controller has two 900GB SAS 2 system boot drives in slots 0 and 1
67. HBAs and six disk shelves in six chains Chapter 2 Hardware Maintenance 175 DE2 to ZS3 4 7420 Clustered fig 5 Three HBAs with multiple disk shelves in six chains Faam Jo af MI 3 210 3 o r r fig 6 Four disk shelves in a single chain DE2 to ZS3 4 7420 Clustered 176 Connecting the ZS3 4 7420 Clustered Controller to Disk Shelves 3 HBAs The following figures show a subset of the supported configurations for Oracle ZFS Storage ZS3 4 7420 clustered controllers with three HBAs Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model Cable Diagrams Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 DE2 to ZS3 4 7420 Clustered fig 3 Three HBAs and three disk shelves in three chains Chapter2 Hardware Maintenance 177 DE2 to ZS3 4 7420 Clustered fig 6 Four disk shelves in a single chain 178 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7120 7320 Cabling 7120 7320 Cabling Connecting 7120 7320 Standalone Controllers to Sun Disk Shelves The following figures show a subset ofthe supported configurations for Oracle ZFS Storage 7120 7320 standalone controllers with one HBA Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller
68. Hardware section ofthe BUI and clicking the drive details icon If you are physically at the system the amber Service Required indicator on the HDD or SSD should be illuminated 2 Ifyouare not physically at the system turn on the locator indicator by clicking the locator x icon 3 Push the release button on the HDD or SSD to open the latch Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 4 Grasp the latch 2 and pull the drive out of the drive slot 5 After 15 seconds navigate to the Hardware gt Maintenance screen and click the details icon on the system controller to verify that the software has detected that the drive is not present 6 Slide the replacement drive into the slot until it is fully seated 7 Close the latch to lock the drive in place The Oracle ZFS Storage system software automatically detects and configures the new drive The device appears in the BUI Maintenance gt Hardware screen when you view details for the controller or drive shelf Fan Module Fan modules are hot swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities Caution Operating a controller for an extended period of time with fans removed reduces the effectiveness of the cooling system For this reason the replacement fan should be unpacked in advance and ready to insert into the controller chassis
69. Maintenance 63 253 2 11 Press the button on the midplane An amber LED indicates the faulty DIMM 12 Press down and out on the release levers at both ends of the DIMM slot 13 Liftthe DIMM up and out ofthe slot Set the DIMM aside on an antistatic mat 14 Open the release levers of the slot where you are installing the DIMM 15 Align the DIMM to the location where it installs into the slot Ensure that the notch in the DIMM lines up with the key in the slot 64 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 16 Insert the DIMM into the slot pressing firmly so that both release levers click closed 17 Lower the drive cage to the horizontal position tighten the four screws and reconnect the cables to the drive backplane and the cable attached to the USB board 18 Replace and secure the air duct 19 Replace the top cover 20 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 21 Connect the power cords to the power supplies 22 Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in 23 Usea pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power butto
70. Manual January 2014 E48492 02 Updates CANCEL CHECK This procedure will consume several minutes and requires a system reboot upon successful update but can be aborted at any time prior to reboot A health check will validate system readiness before an update is attempted and may also be executed independently by clicking the Check button System ready Checking as Q Deferred Updates After completion the System Ready state changes to Yes or No with a link to the Alert Log This procedure will consume several minutes and requires a system reboot upon successful update but can be aborted at any time prior to reboot A health check will validate system readiness before an update is attempted and may also be executed independently by clicking the Check button System ready No Alert tog Deferred Updates CLI To execute the update health checks via the CLI execute the upgrade command in maintenance system updates after selecting the update media dorab maintenance system updates ak nas 2013 06 05 0 0 1 1 6 gt upgrade This procedure will consume several minutes and requires a system reboot upon successful update but can be aborted with Control C at any time prior to reboot A health check will validate system readiness before an update is attempted and may also be executed independently using the check command Are you sure Y N Healthcheck running Healthcheck completed There are no issues at
71. NIC 4 Optional recommended UTP front end 3 7051223 2 port IOGbE optical NIC 6 Optional recommended front end 3 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 4 7070006 2 port 1OGbE copper NIC 4 Optional recommended front end 4 375 3696 01 2 portInfiniband CX2 HCA 4 Optional recommended front end 4 7070195 4 port IGbE copper NIC 4 Optional recommended UTP front end 4 7051223 2 port IOGbE optical NIC 6 Optional recommended front end 4 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup C 511 1496 05 Cluster Controller 200 1 Cluster base configuration Chapter 2 Hardware Maintenance 49 Z53 4 50 Slot Mfg Part Number Description Max Note 5 7070006 2 port IOGbE copper NIC 4 Optional recommended front end 5 375 3696 01 2 portInfiniband CX2 HCA 4 Optional recommended front end 5 7070195 4 port IGbE UTP Ethernet 4 Optional recommended front end 5 7051223 2 port 10GbE optical NIC 6 Optional recommended front end 5 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 6 7070006 2 port 1OGbE copper NIC 4 Optional recommended front end 6 375 3696 01 2 portInfiniband CX2 HCA 4 Optional recommended front end 6 7070195 4 port IGbE copper NIC 4 Optional recommended UTP front end 6 7051223 2 port 10GbE optical NIC 6 Optional recommended front end 6 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 7 7067
72. Number Description FRU CRU F371 4966 01 DIMM 8GB DDR3 2RX4 13 CRU F371 4885 01 Intel E5620 2 40G FRU All sockets must be occupied by either a filler or a DDR3 DIMM All DDR3 DIMMs must be identical DIMMs are pre installed in PO slots D1 D2 D4 D5 D7 and D8 D6 D7 D D5 D4 03 D2 D1 Do Do D1 D2 03 D4 D5 De D7 DB Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7120 Power Distribution Fan Module and Disk Components The fan modules and LEDs are shown in the following illustration Figure Legend 1 Fan Board 5 Power Distribution Board Chapter 2 Hardware Maintenance 97 7120 98 Figure Legend 2 SAS Expander Board 6 Connector Board 3 Disk Backplane 7 Power Supply Backplane 4 Front Control Panel Light Pipe Assembly Standalone Controller Configurations The following table shows the configuration options for a 7120 controller All PCIe cards are low profile and must be fitted with low profile mounting brackets This table describes base configurations for the 7120 with Aura Logzilla Mktg Part Number Description Mfg Part Number TA7120 12TB 7120 1xCPU 24GB I2TB 597 0754 01 TA7120 24TB 87120 1xCPU 24GB 24TB 597 0755 01 The following table describes base configurations for the 7120 with Logzilla 3 5 SSD Mktg Part Number Description Mfg Part Number 7101282 7120 1xCPU 24GB 3 3TB 7014523 7101284 712
73. Oracle ZFS Storage Appliance Customer Service Manual e Sun ya Copyright 2009 2014 Oracle and or its affiliates All rights reserved This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws Except as expressly permitted in your license agreement or allowed by law you may not use copy reproduce translate broadcast modify license transmit distribute exhibit perform publish or display any part in any form or by any means Reverse engineering disassembly or decompilation of this software unless required by law for interoperability is prohibited The information contained herein is subject to change without notice and is not warranted to be error free If you find any errors please report them to us in writing Ifthis is software or related documentation that is delivered to the U S Government or anyone licensing it on behalf of the U S Government the following notice is applicable U S GOVERNMENT END USERS Oracle programs including any operating system integrated software any programs installed on the hardware and or documentation delivered to U S Government end users are commercial computer software pursuant to the applicable Federal Acquisition Regulation and agency specific supplemental regulations As such use duplication disclosure modification and adaptation of the programs including any
74. RU F300 2233 02 RoHS 760W Power Supply CRU F541 2075 04 Buss Bar Power 1U FRU F542 0184 01 DR 3GbSATA CRU F542 0330 01 2 5 512GB ReadZillaSSD CRU Chapter 2 Hardware Maintenance 105 7320 106 Part Number Description FRU CRU F541 276 01 ASSY FAN Module CRU F541 4274 02 Fan Board 1U FRU 7320 PCle Cards and Risers Following is the complete list of replaceable PCIe cards for the 7320 system Part Number Description FRU CRU F371 4325 01 8Gb FC HBA PCIe CRU F375 3609 02 PCA SAS 6GBS 8 Port PCIe CRU F375 3606 03 Dual Port x4 IB HCA PCIe CRU F375 3696 01 Dual Port CX2 4XQDR PCle CRU F375 3617 01 2X10GbE SFP X8 PCIe CRU F375 3481 01 NIC Card Quad Port 1GigE Cu PCIe CRU F511 1496 04 Sun Fishworks Cluster Controller 200 PCIe FRU 7320 Rear Panel Following is an illustration ofthe 7320 storage controller rear panel The Sun 375 3609 belongs in slot 2 cannot be installed in any other slots and a second is not offered as an option Figure Legend 1 Power supplies 4 Network management port 2 SC summary status LEDs 5 Ethernet ports 3 Serial management port 6 PCIe slots Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7320 7320 Single and Cluster Controller Configurations The single controller base configuration is 96GB RAM 2x2 4GHz Quad Core processors one external SAS HBA and four 10 100 100
75. S3 2 controller motherboard memory and PCIe components are described in the following figure and legend Note Field replaceable components FRUs are not customer serviceable and must be replaced by trained Oracle service technicians 36 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 Figure Legend 1Drive cage FRU 5 Airduct 9 Battery 2 Drive backplane FRU 6PCleRisers FRU 10 PDB duct 3 Motherboard FRU 7 PCIe cards 11 Power distribution board FRU 4 DIMMs 8 USB flash drive CPU and Memory The ZS3 2 controller motherboard has 16 slots in two groups that hold industry standard DDR3 low voltage LV DIMMS Chapter 2 Hardware Maintenance 37 253 2 P1 D3 PO D3 KC ets PO D4 DIMM 13 0 Cp P0 D5 b 2 amp IPi ICPU 0 DIMM 14 0 SS Pops DIMM 15 0 SST y p0 D7 MM 10 0 SST po p2 MM 11 0 C E B Pr DIMM 0 0 puri P1 D0 p SD P1101 DIMM_1 2 DIMM 2 0 pu D 1 2 DIMM 3 0X p rDy DIMM 128 ID ID ID ID ID ID ID ID Note All sockets must be occupied by a DDR3 DIMM or a filler All DDR3 DIMMs must be identical Capadty CPUO CPU 1 256GB D0 D2 D5 D7 blue DO D2 D5 D7 blue D1 D3 D4 D6 white D1 D3 D4 D6 white The ZS3 2 controller replaceable memory components and part numbers are listed below Component Description FRU
76. Storage ZS3 4 controllers Refer to Disk Shelf Details for replacing expansion storage shelf components Prerequisites Read the information in the overview section for your controller to become familiar with the replaceable parts of the system ZS3 4 Overview Follow the instructions in the Safety Information and Required Tools and Information sections Safety Information This section contains safety information that you must follow when servicing the storage system For your protection observe the following safety precautions when setting up your equipment Donotremove the side panels or run the storage system with the side panels removed Hazardous voltage is present that could cause injury The covers and panels must be in place for proper air flow to prevent equipment damage Follow all cautions warnings and instructions marked on the equipment and described in Important Safety Information for Oracle s Hardware Systems Ensure that the voltage and frequency of your power source match the voltage inscribed on the electrical rating label Follow the electrostatic discharge safety practices Electrostatic discharge ESD sensitive devices such as PCI cards HDDs SSDs and memory cards require special handling Circuit boards and HDDs contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static electricity from clothing or the work environment can destroy the components
77. UI and click the details icon on the controller and then click Slots to locate the faulted component Caution This procedure requires that you handle components that are sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components All HBAs must be of the same type Ensure that you upgrade your system software before installing a newly released HBA You must shut down the controller before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one ofthe following methods Log in to the BUI and click the power icon on the left side of the masthead SSH into the storage system and issue the maintenance system poweroff command SSH or serial console into the service processor SP and issue the stop SYS command Use a pen or non conducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown wherein all applications and files will be closed abruptly without saving press and hold the power button for at least four seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller is in standby power mode Disconnect the AC power cords from the rear panel of the storage controller Verify that no cables will be damaged or will interf
78. ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 DIMM names in appliance logs and the Maintenance gt Hardware view are displayed with the full name such as SYS MB P0 D7 Fan Modules The Fan Modules and Fan Module LEDs ofthe controller are shown in the following figure The following LEDs are lit when a fan module fault is detected E3107 310 3 HIE t J m Frontand rear Service Action Required LEDs Fan Module Service Action Required TOP LED on the front of the server a Fan Fault LED on or adjacent to the faulty fan module The system Overtemp LED might light ifa fan fault causes an increase in system operating temperature NIC HBA Options This table describes NIC HBA PCIe card options for ZS3 4 standalone and cluster configurations Mktg Part Number Description SG SAS6 INT Z 8 port 6Gb s SAS 2 internal HBA SG XPCIE2FC QF8 Z 2 port 8Gb FC HBA 7103791 4 port 4x4 SAS 2 6Gb s external HBA 7100477 4 port 1Gb Ethernet Copper UTP Chapter 2 Hardware Maintenance 47 Z53 4 48 Mktg Part Number Description 7100488 2 port 10Gb Ethernet Copper Base T X4242A 2 port Infiniband CX2 HCA X1109A Z 2 port LOGbE SFP NIC X2129A Transceiver 850NM 1 10GPS Short Reach SFP X5562A Z Transceiver 1OGbE 1GbE Long Reach SFP Connectors The serial management connector SER MGT is an RJ 45 connector and provides a terminal connection to the SP c
79. able Management Arm The following figure identifies the components of the second generation cable management arm CMA See the CMA installation instructions 40 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 ZS3 4 Figure Legend 1 Connector A 6 Connector D 2Frontslidebar 7 Slide rail latching bracket used with connector D 3 Velcro straps 6 8 Rear slide bar 4 Connector B 9 Cable covers 5 Connector C Attached Storage The ZS3 2 controller connects to external storage through a 4 port 4x4 SAS 2 6Gb s HBA Youcan attach from one to eight HDD only or SSD HDD Logzilla capable disk shelves to the controller You can also attach mixed disk shelf types DE2 and Sun Disk Shelf in the same chain For more information see Connecting to Attached Storage ZS3 4 Hardware Overview Use the information on this page as a preparation reference for servicing replaceable components of the Oracle ZFS Storage ZS3 4 controller Refer to the following topics for procedural instructions Controller Tasks replace system controller components Disk Shelf Tasks replace disk shelf components Controller Overview The ZS3 4 controller can be configured as a single controller or two controllers to create a high availability cluster configuration The following table describes the configuration options Mktg Part Software Number CPU Memory Readzilla SAS 2 Boot Drive SAS 2
80. acket alignment pin and out ofthe socket 10 Swing the retainer to its fully open 120 degrees position 11 Position the PCIe card with where it will install into the riser The PCIe card bracket mates with the alignment pin and the socket of the riser 12 Press the PCIe card into the socket 68 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 13 Swing the retainer to its fully closed position If there is resistance check the alignment of the PCIe card bracket and try again 14 Align the riser to the location where it installs into the chassis 15 Reconnect any previously removed internal cables to any PCIe cards installed in the riser if doing so will ease installation of the riser 16 Lower the riser card onto the motherboard and press the card edge connector securely into the socket The bracket ofthe riser overlaps the bracket of the adjacent riser 17 Tighten the two captive screws 18 Reconnect any previously removed internal or external cables to any PCIe cards installed in the riser 19 Replace the top cover 20 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller Chapter2 Hardware Maintenance 69 253 2 21 22 23 24 25 26 Connect the power cords to the power supplies Verify that standby power is on indicate
81. after refreshing the hardware view this may be an indication ofa problem with the device Check the Maintenance Problems view for any relevant faults that may have been identified Additionally in some cases the controllers in the disk shelves may remain offline during firmware upgrade If this occurs no other controllers will be updated until this condition is fixed If an enclosure is listed as only having a single path for an extended period of time check the physical enclosure to determine whether the green link lights on the back of the SIM are active If not remove and re insert the SIM to re establish the connection Verify that all enclosures are reachable by two paths Rollback The rollback procedure reverts all of the system software and all of the metadata settings of the system back to their state just prior to applying an update This feature is implemented by taking a snapshot of various aspects of the system before the new update is applied and rolling back this snapshot to implement the rollback The implications of rollback are as follows Any appliance configuration changes are reverted and lost For example assume you are running version V and then you update to V 1 and then you change your DNS server If you execute a rollback then your DNS server setting modification is effectively undone and removed from the system permanently Conversely any changes made to user data are not reverted if you update from V to V 1
82. aintenance 71 253 2 72 6 Lift the battery out of the receptacle and set it aside 7 Position the battery where it will install into the receptacle with the positive side of the battery facing up 8 Press the battery into the receptacle until you hear a click 9 Install Riser 1 10 Install the top cover 11 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 12 Connect the power cords to the power supplies 13 Usea pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit 14 Connect data cables to the PCIe card routing them through the cable management arm 15 When the system has finished booting log in and set the time using the steps in the BUI Clock task Oracle DE2 and Sun Disk Shelf Interoperability Oracle DE2s and Sun Disk shelves can be used together with standalone and clustered ZS3 2 controllers after certain hardware modifications Contact Oracle Service to update your controllers for disk shelf interoperability Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 ZS3 4 ZS3 4 CRU Maintenance Procedures This section provides instructions on how to replace customer replaceable components CRUs for Oracle ZFS
83. ally applying them If an update has not yet been applied i e is not running and is not a rollback target it can be deleted via either the BUI or the CLI You might want to delete images in order to free up needed space to download new images Administrators should verify that the system is in a healthy state prior to applying the update The details are described below in the preconditions section After the media is unpacked and verified the update can be applied During this process an update health check will be performed to verify the appliance is ready to update You may be asked to set update options and confirm For more information on these questions see the section on deferred updates If the update is no longer appropriate for the system because you have skipped past its version number an error message may be provided During the update messages and a progress meter will appear to indicate that the update is proceeding The installation portion of the update will take about half an hour to complete however the full upgrade process may not be complete at that point See below regarding additional firmware upgrades that may take place following the reboot While the upgrade is in progress up until the reboot and following the reboot during any firmware upgrades it is non disruptive the controller continues to provide data services to clients If the system software fails during the upgrade it will reboot and continue run
84. alone controller with two HBAs and one disk shelf in a single chain Hd ab r m 333 d ont Ha v ed zm aie e d fig 4Standalone controller with six HBAs and one disk shelfin a single chain Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 Cabling fig 8 Six HBAs and 36 disk shelves in six chains Chapter2 Hardware Maintenance 183 7420 Cluster Cabling 7420 Cluster Cabling 184 Connecting the 7420 Clustered Controllers to Disk Shelves 2 and 6 HBAs The following figures show a subset of the supported configurations for Oracle ZFS Storage 7420 clustered controllers with two and six HBAs Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model Cable Diagrams al 7420 3 8 NU 7420 fig 1 Clustered controllers with two HBAs and one disk shelf in a single chain T m 7420 fig 2 Two HBAs and two disk shelves in two chains bea Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 Cluster Cabling fig 5 Six HBAs and two disk shelves in two chains Chapter2 Hardware Maintenance 185 7420 Cluster Cabling
85. an module status LEDs and transfer I2C data for the fan modules m Disk Backplane The hard drive backplane is a FRU and includes the connectors for the hard disk drives as well as the interconnect for the I O board Power and Locator buttons and system component status LEDs The storage controller has an eight disk backplane Each drive has an LED indicator for Power Activity Fault and OK to Remove not supported The following list contains the replaceable system boards for the 7320 storage controller Part Number Description FRU CRU F541 2883 01 X8 PCIe Riser Card 1U CRU F541 2885 01 X16 PCIe Riser Card 1U CRU F541 4081 01 RoHS Motherboard and Tray FRU F511 1489 01 DB Power Distribution Board FRU F511 1548 01 PCB 8 Disk 1U Backplane FRU F541 4275 02 PCBA Connector Board 1U FRU Cables The following list contains the replaceable cables for the 7320 storage controller Part Number Description FRU CRU F530 4228 01 Cable MiniSAS FRU internal F530 3927 01 FRU CBL PDB MB 1U 2U RIBBON FRU internal F530 4431 01 Cable Fan data FRU internal F530 4417 01 FRU Cable Fan paddle FRU internal F530 3880 01 Cable Assembly Ethernet Shielded RJ45 RJ45 6m CRU external F530 3883 01 FRU 2M 4X Mini SAS Cable CRU external Chapter 2 Hardware Maintenance 101 7320 73201 0 Components The following figure and legend identify the I O
86. appliance using one of the following methods Login to the BUI and click the power icon on the left side of the masthead SSH into the storage system and issue the maintenance system poweroff command SSH or serial console into the service processor and issue the stop SYS command Useapen or non conducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown wherein all applications and files will be closed abruptly without saving press and hold the power button for at least four seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller isin standby power mode 1 Disconnect the AC power cords from the rear panel of the storage controller 2 Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack 3 From the front of the storage controller release the two slide release latches 4 While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch 5 To remove the top cover Chapter 2 Hardware Maintenance 87 Z53 4 Simultaneously lift both cover latches in an upward motion 6 Liftup and remove the top cover 7 Usinga small non metallic screwdriver press the latch and remove the battery from the motherboard The battery is shown here 8 Press the replacement battery into the motherboard wi
87. aque binary downloads that contain some or all of Management and system software Firmware for internal components such as HBAs and network devices Firmware for disks and flash devices Firmware for external storage enclosure components In general the update release notes describe what is in the update and the update process automates all of the steps of activating the delivered components Procedure Overview The procedure for updating the system is as follows First the software update media is downloaded from an Oracle support website or retrieved from another official source The media is represented by a single compressed file named after the version number such as ak nas 2013 06 05 0 0 0 pkg gz The file can be renamed if needed as the true version number is recorded internally within the image The compressed media packages will vary in size but typically will be on the order of several hundred megabytes Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates Second the software media is uploaded to the appliance This can be done via either the BUI or the CLI see below for details of this operation After the media is uploaded it will be unpacked and verified If all verification checks pass it will appear in the list of update images as eligible for installation Any number of images can be maintained on the appliance subject to a system disk space quota without actu
88. ard options for standalone and cluster configurations Mktg Part Number Description Mfg Part Number SGX SAS6 INT Z 8 port 6Gb s SAS 2 internal HBA 7054001 SG XPCIE2FC QF8 Z 2 port 8Gb FC HBA 371 4325 02 SG XPCIESAS GEN2 Z 2 port SAS x4 external HBA F375 3609 03 7105394 2 port SAS x4 external HBA for EU countries 7059331 X4446A Z N 4 port GigE UTP 7054739 X4242A 2 port CX2 4xQDR HCA 594 6776 01 X1109A Z 2 port LOGbE SFP NIC 7051223 X2129A XCVRm 850NM 1 10GPS Short Reach SFP 7015839 X5562A Z 10GbE 1GbE SFP Transceiver LR 594 6689 01 PCle Options The 7420 controller has ten PCIe slots This table describes base and optional PCIe card slots for standalone and cluster configurations Note The 7420 M2 reserves slot 0 for a SAS 2 internal HBA and slots 1 and 8 for two SAS 2 external HBAs Slot Mfg Part Number Description Max Note 0 7054001 8 port 6Gb s SAS 2 internal 1 Slot 0 of the 7420 M2 is reserved for the SAS 2 internal HBA HBA 0 371 4325 02 8Gb 2 port FC HBA 6 Optional FC target or initiator Backup for the 7420 controller 0 7051223 2 port optical 1OGbE NIC 6 Optional recommended front end for the 7420 controller 0 7054739 4 port copper NIC 6 Optional recommended front end for the 7420 controller 1 F375 3609 03 2 port SAS external HBA 6 Base configuration 2 minimum 7059331 for EU countries 2 F375 3609 03 2 port SAS external HBA 6 Additional optional back end 70
89. ations PGRs Before this upgrade has been applied the number of reservations stored with each LUN is severely limited and may even be zero Applying this update is equivalent to upgrading the on disk ZFS pool to version 16 Triple Parity RAID Dedup Triple Parity RAID Deferred Update This update provides the ability to use the triple parity RAID storage profile RAID Z with three parity sectors per stripe Triple parity offers increased protection against drive failures and additional overall availability In order to use this new mode the storage pool must be upgraded If you choose not to upgrade the pool and attempt to use this new property you will get an error indicating that the storage pool needs to be upgraded first There is no other implication of applying this update and it can be ignored if there is no need to use this new setting Applying this update is equivalent to upgrading the on disk ZFS pool to version 17 Data Deduplication Deferred Update This update provides the ability to use data deduplication Chapter3 System Maintenance 215 Replication In order to use this new mode the storage pool must be upgraded If you choose not to upgrade the pool and attempt to use this new property you will get an error indicating that the storage pool needs to be upgraded first There is no other implication of applying this update and it can be ignored if there is no need to use this new setting Applying this update
90. atures The screen allows the administrator to View the status ofthe system disks Manage software updates and update the system software Create and restore appliance configuration backups Create and upload a support bundle Repeat the initial setup with existing settings Reset the system to the factory defaults View pending disk firmware updates System Disks The system disks section shows the status ofthe system disks and their current usage The BUI displays this with a pie chart and the CLI as a text list For example tarpon maintenance system disks show Properties profile mirror root 1 14G var 52 4M update 2 52M stash 14 8M dump 16 0G cores 18K unknown 39 0G free 401G 193 System 194 Disks DISK LABEL STATE disk 000 HDD 7 healthy disk 001 HDD 6 healthy Note The disk column is not required by the GUI Support Bundles The appliance can generate support bundles containing system configuration information and core files for use by remote support in debugging system failures Support bundles are generated automatically in response to faults if the Phone Home service is enabled Administrators can manually generate and upload a support bundle from this section ofthe Maintenance System screen Once generated support bundles are automatically uploaded to Oracle s Support files Service at http support oracle com http support oracle com To facilitate this the appliance m
91. boards Do not touch the components without using antistatic precautions especially along the connector edges Required Tools and Information The following tools are needed to service the CRUs Antistatic wrist strap Wear an antistatic wrist strap and use an antistatic mat when handling components such as HDDs or PCI cards When servicing or removing storage controller components attach an antistatic strap to your wrist and then to a metal area on the chassis Following this practice equalizes the electrical potentials between you and the storage controller Antistatic mat Place static sensitive components on an antistatic mat No 2 Phillips screwdriver Nonconducting No 1 flat blade screwdriver or equivalent Nonconducting stylus or pencil to power on the storage controller Chassis Serial Number To obtain support for your storage controller or to order new parts you need your chassis serial number You can find a chassis serial number label on the storage controller front panel on the left side Another label is on the top ofthe storage controller Alternatively click the Oracle logo in the BUI masthead to obtain the serial number or issue the following command hostname maintenance hardware show Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 Prepare the Controller for Servicing Internal Components ZS3 2 Prepare the Controller for Servicing Internal Components Remove Power
92. bold italics m Help is always available the help command provides context specific help Help on a particular topic is available by specifying the topic as an argument to help for example help commands Available topics are displayed by tab completing the help command or by typing help topics You can combine these two principles as follows dory gt help tab builtins commands general help properties script Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Hardware Locating a disk Hardware View The Maintenance Hardware screen also known as the hardware view provides component status of the appliance and attached disk shelves This information is available from both the BUI and the CLI BUI The BUI hardware view provides interactive illustrations that enable you to browse through the appliance and attached disk shelf components The screenshot at the top of this page shows a disk highlighted in a Sun ZFS Storage 7320 showing both its physical location and details The buttons in the hardware view are icon description icon description Showa Xy Toggle blinking ofthe locator LED for this component more id detailed view of this component Leave this I Power off reboot or diagnostic reboot detailed view Chapter 1 Introduction 15 Hardware 16 icon description icon description e Click for Qo Offline disk more details v Hardware WW
93. ce at any time For details refer to the Oracle Software License Agreement SLA and Entitlement for Hardware Systems with Integrated Software Options RAID Z RAID 5 and RAID 6 mirrored and striped disk configurations Unlimited read only and read write snapshots with snapshot schedules Data deduplication Built in data compression Remote replication of data for disaster recovery Active active clustering for high availability Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Overview Thin provisioning of iSCSI LUNs Virus scanning and quarantine NDMP backup and restore Availability To maximize the availability of your data in production Oracle ZFS Storage appliances include a complete end to end architecture for data integrity including redundancies at every level of the stack Key features include Predictive self healing and diagnosis of all system hardware failures CPUs DRAM I O cards disks fans power supplies ZFS end to end data checksums of all data and metadata protecting data throughout the stack a RAID 6 double and triple parity and optional RAID 6 across disk shelves Active active clustering for high availability Linkaggregations and IP multipathing for network failure protection O Multipathing between the controller and disk shelves Integrated software restart of all system software services Phone Home of telemetry for all software and
94. ce to clients as the active networking configuration and data protocols are reconfigured Therefore a configuration restore should only be used on a development system or during a scheduled downtime Service interruption Clients accessing data on the system through a data protocol such as NFS will see service interrupted as the network is reconfigured and the NFS service restarted If the selected backup copy was taken when a service was disabled by the administrator that setting will be restored and therefore client sessions will be terminated for that protocol m Session interruption If restore is initiated from a web browser that web browser session will also be disconnected during the restore process as the network is reconfigured Ifthe restored configuration does not include the same routing and network address settings used by the current browser connection or if the browser is connected to a network address managed by DHCP the browser session will be interrupted during the restore The restore process will complete in the background but you will need to reload or point the browser at a new restored network address to continue For this reason it may be desirable to initiate a complex configuration restore from the service processor serial console using the CLI Un cluster restore and re cluster Configuration backups may be initiated for appliances that are joined in a cluster but a configuration restore may not be used
95. components ofthe storage controller Figure Legend 1Top cover 2 Left Control Panel Light Pipe Assembly 3 Drive Cage 4 Solid State Drives 5 blank USB Module 6 Right Control Panel Light Pipe Assembly 7320 CPU and Memory Components The following list contains the replaceable CPU and memory components ofthe 7320 Part Number Description FRU CRU F371 4966 01 DIMM 8GB DDR3 2RX4 13 CRU F371 4885 01 Intel E5620 2 40G FRU 102 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7320 The storage controller motherboard has 18 slots in two groups that hold industry standard DDR3 DIMM memory cards All sockets must be occupied by either a filler ora DDR3 DIMM D8 D7 06 D5 D4 D3 D D1 Do Do D1 D2 D3 D4 05 be D7 D8 7320 Power Distribution and Fan Module Components The following figure and legend identify the Power Distribution Fan Module components of the storage controller Figure Legend 1Fan Modules 4Power Distribution Bus Bar Assembly 2 Fan Board 5 Power Supplies Chapter 2 Hardware Maintenance 103 7320 104 Figure Legend 3 Paddle Board Electrical Specifications The following list shows the electrical specifications for the 7320 Note The power dissipation numbers listed are the maximum rated power numbers for the power supply The numbers are nota rating of the actual power consumption ofthe appliance Connectors Two C13 conn
96. d by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in Usea pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit Connect data cables to the PCIe card routing them through the cable management arm Go to the Maintenance Hardware screen ofthe BUI and click the details icon amp on the controller Then click Slots to verify the status ofthe new component The status indicator should appear green Install the disk shelf and connect the expansion storage USB Flash Drive Removing the USB flash drive is a cold service operation You must power off the appliance before you remove the USB There will be a loss of access to the storage unless the system is in a clustered configuration The USB flash drive is located in the forward most USB socket at the center rear of the motherboard labeled USB 1 near where Riser 2 installs 1 2 3 4 5 Power off the storage controller Extend the storage controller from the rack Remove the top cover and air filter Ifa PCIe card installed in Riser 2 prevents you from accessing the USB drive remove Riser 2 Grasp the USB drive and pull it straight up and out of the chassis 70 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 6
97. done Updating system logs done Starting primordial svc configd Upgrading SMF repository This may take several minutes Upgrading from Version 5 to Version 6 11570 of 11570 rows upgraded Upgrading from Version 6 to Version 7 6305 of 6305 rows upgraded Upgrading from Version 7 to Version 8 Chapter 3 System Maintenance 203 Updates 204 SMF repository upgrade complete SMF online in 180 seconds Sanitizing manifestfiles properties done Loading smf 5 service descriptions 162 162 svccfg Loaded 162 smf 5 service descriptions Transitioning NFS server properties done Re enabling auditing of Solaris commands done Transitioning network initial IPMP properties to network ipmp done Transitioning name service properties done Transitioning CIFS server properties done Preparing for service import done Importing adconf xml done Configuring appliance kit identity default done Applying service layer ak generic done Refreshing services done Applying service layer ak nas done Refreshing services done Applying service layer ak SUNW iwashi plus done Refreshing services done Applying service profile ak generic done Applying profile upgrade akinstall xml done Applying layer upgrade composite svc done Cleaning up services done Shutting down svc configd done Configuring devices Configuring network devices Sun ZFS Storage 7120 V
98. e Power off the storage controller 4 Extend the storage controller from the rack 5 Remove the top cover and air filter Chapter2 Hardware Maintenance 59 253 2 6 Grasp the green band ofthe fan module to be removed with your forefinger and press down on the fan module with your thumb 7 Pulltheband straight up and lift the fan module out ofthe chassis 8 Align thereplacement fan module into the fan tray slot The green band is on the top ofthe fan module and the arrow points to the rear of the chassis 60 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 9 Lower the fan module into the fan tray and press down until the module clicks securely in the fan tray 10 Replace the top cover 11 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 12 Connect the power cords to the power supplies 13 Usea pen or other pointed object to press and release the recessed Power button on the storage controller front panel 14 Verify that the Fan OK status indicator is lit and that the fault status indicator on the replaced fan module is dim Memory Removing a DIMM is a cold service operation You must power offthe appliance before you remove the DIMM There will be a loss of access to the storage unless the system is in a clustered configuration To identify a g
99. e adding disk shelves to an individual chain Balance the number of disk shelves across the disk chains in your system Disks with different capacities or rotation speeds cannot be mixed within a single disk shelf To use mixed disk shelf types the controller must have only 4X4 port SAS 2 HBAs Do not use mixed disk shelves in the same chain each must be on their own SAS fabric The following list provides links to cable diagrams for standalone and clustered controllers Oracle DE2 24 Disk Shelf Connecting DE2 Disk Shelves to the ZS3 2 7120 7320 ZS3 2 7320 Clustered Connecting DE2 Disk Shelves to the ZS3 4 7420 ZS3 4 7420 Clustered Sun Disk Shelf Connecting Sun Disk Shelves to the 7120 7320 7320 Clustered Connecting Sun Disk Shelves to the 7420 7420 Clustered Mixed DE2 24 and Sun Disk Shelves Connecting mixed DE2 24 and Sun Disk Shelves to the ZS3 2 7120 7320 ZS3 2 7320 Clustered Connecting mixed DE2 24 and Sun Disk Shelves to the ZS3 4 7420 ZS3 4 7420 Clustered Next Steps After connecting attached storage apply power and configure the appliance Poweringon Initial configuration Storage expansion may require adding or replacing HBAs in your configuration For information see the following sections ZS3 2Maintenance Procedures PCIe Cards 7ZS3 4 Maintenance Procedures PCIe Cards and Risers 7x20 Maintenance Procedures PCle Cards and Risers Chapter 2 Hardware Maintenance 169
100. e process of reverting the system to that saved configuration Review the Restore Impact guidance above and confirm that it is ok to proceed Delete a Saved Configuration To delete a Saved Configuration simply click the TII trash can icon to delete the configuration that is no longer required Export a Saved Configuration To export a Saved Configuration mouse over the configuration list entry you wish to export and click the m download icon Your browser will prompt you to save the file locally The file is a compressed archive whose contents are versioned and may vary over time You should not attempt to unpack or modify the content ofthe archive and doing so will render it unable to be imported back to the appliance successfully Import a Saved Configuration To import a previously exported Saved Configuration click the amp add icon at the top ofthe saved configurations list and then use your web browser s file selection dialog to locate the previously exported configuration You should upload the single compressed archive file previously saved using the export function Managing Configuration Backups Using the CLI The following section outlines how various Configuration Backup tasks can be accomplished using the CLI in the maintenance system configs context Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ConfigurationBackup Listing Configurations host maintenance system configs list
101. e worst case the upload will proceed all the way to completion and you may not see the progress bar Chapter3 System Maintenance 209 Updates 210 Unpacking and Verifying Media This step will happen automatically after the media is done uploading Available Updates VERSION RELEASE DATE STATUS unpacking unpacking Unpacking update 2008 10 10 1 0 2008 10 10 01 45 Previous systom software gt 2008 10 14 1 0 2008 10 14 16 10 49 Current system software Beginning the Upgrade After the update is uploaded unpacked and verified it will appear as an update Available Updates VERSION RELEASE DATE STATU 2000090130 1 18 Ubbededat 2000120 52519 z Click the information icon to view the Release Notes for the software update To begin the upgrade click on the amp apply icon During this process an update health check will be performed to verify the appliance is ready to update As the upgrade progresses you will see the most recent message in the status field of the update To cancel the update at any time and without ill effect click on the cancel icon Rolling Back To roll back locate a previous image and click on the rollback icon You will be asked to confirm that you wish to execute a rollback and then the system will reboot and execute the rollback Unlike fail safe rollback you will not be asked for further confirmation when the system reboots Removing Update Media To rem
102. ear ofthe storage controller byaboutan inch 2 5 cm Slide the top cover toward the rear of the chassis 2 until it seats Chapter2 Hardware Maintenance 141 7x20 142 Press down on the cover with both hands until both latches engage 12 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 13 Connect the power cords to the power supplies 14 Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in 15 Usea pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit 16 Connect data cables to the PCIe card routing them through the cable management arm 17 When the system has finished booting log in and set the time using the steps in the BUI Clock task Oracle DE2 and Sun Disk Shelf Interoperability Oracle DE2s and Sun Disk shelves can be used together with standalone and clustered 7x20 controllers after certain hardware modifications Contact Oracle Service to update your controllers for disk shelf interoperability Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Shelf Disk Shelf Overview Oracle disk shelves are high
103. ease button toward the lever hole to release the lever N Chapter2 Hardware Maintenance 163 Shelf 164 Grasp the lever and remove the I O Module being careful not to damage the connector pins in back With the lever of the new I O Module in the open position slide the I O Module into the disk shelf being careful of the connector pins Push the lever fully closed until you hear or feel a click Reconnect the interface cables to their original locations Wait approximately 60 seconds for the I O Module to complete its boot process at which time the Power LED should be solid green and the Fault Locate LED should be off All four activity LEDs should be solid green for each SAS 2 port that has an interface cable connected to it Goto the Maintenance Hardware section of the BUI click the right arrow icon Q atthe beginning of the appropriate disk shelf row and then click Slot to verify that the I O Module icon is green for the newly installed I O Module Replacing a SIM Board The following disk shelves have SIM boards Sun Disk Shelf The SIM boards which are similar to I O Modules are hot swappable so you can replace them without removing power to the system The SIM boards are multi pathed so you can remove one of the SIM boards at any time regardless of the state of the blue SIM OK indicator A faulted SIM board is indicated by an amber LED Go to the Maintenance gt Hardware section of the BUI click
104. ectors which work on 110 220v outlets Input Nominal frequencies 50 60Hz Nominal voltage range 100 120 200 240 VAC Maximum current AC RMS 9 0 amps Max AC operating range 90 264 VAC Output 3 3 VDCSTBY 3 6A m 12 VDC 62 3A Power dissipation Max power consumption 873 W Max heat output 2977 BTU hr Volt Ampere rating 891 VA 240 VAC 0 98P F 7320 Front Panel The following figure and legend identify the front panel LEDs Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7320 Figure Legend 1 Locate Button LED 2 Service Required LED amber 3 Power OK LED green 4 Power Button 5 Rear Power Supply 6 System Overtemperature LED 7 Top Fan The following figure and legend identify the 7320 front panel drive locations Two mirrored hard disk drives HDDs that store the operating system reside in slots 0 and 1 Up to four solid state drives ReadZilla SSDs which store the read cache fill slots 2 through 5 in order Slots 6 and 7 are empty and must contain drive fillers 66 60 Disk Drive Locations HDD1 HDD3 HDD5 HDDO HDD2 HDD4 HDD6 HDD7 7320 Replaceable Components The following list contains all of the replaceable power distribution disk and fan module components of the 7320 Note that power supplies disks and fan modules are hot pluggable on the storage controller Part Number Description FRU C
105. ed front end 9 7054739 4 port copper NIC 6 Optional recommended front end PCle Slot Order When installing additional PCIe cards to your base 7420 configuration cards must be added in a specific order Note The 7420 M2 reserves slot 0 for a SAS 2 internal HBA and slots 1 and 8 for two SAS 2 external HBAs see PCIe Options Install optional PCIe cards in the following order Chapter2 Hardware Maintenance 119 7x20 7x20 120 PCle Card Slot Order 1 SAS 2 external HBA Slot 2 7 3 6 2 8Gb FC HBA Slot 9 Additional FC HBAs are installed last 3 Infiniband HCAs Slot 7 2 6 3 5 4 4 10Gb Ethernet Slot 7 2 6 3 5 4 0 not available for 7420 M2 9 5 Quad 1Gb Ethernet Slot 7 2 6 3 5 4 0 not available for 7420 M2 9 6 Remaining 8Gb FC HBAs Slot 7 2 6 3 5 4 0 not available for 7420 M2 Attached Storage Primary storage for 7420 controllers is provided by one to six chains of attached storage Each chain consists ofup to six HDD only or SSD capable disk shelves combined in any order See Disk Shelf Overview for component specifications and diagrams 7x20 CRU Maintenance Procedures This section provides instructions on how to replace customer replaceable components CRUs in Oracle s Sun ZFS Storage 7120 7320 and 7420 controllers Refer to Disk Shelf Details for replacing expansion storage shelf components Prerequisites Read the information in the overview section fo
106. elf interoperability Chapter 2 Hardware Maintenance 89 7120 7120 90 7120 Hardware Overview Use the information in this section as a reference when preparing to service replaceable components of the Sun ZFS Storage 7120 Refer to the following for procedural instructions Controller Tasks replace system controller components m Disk Shelf Tasks replace disk shelf components Chassis Overview The Sun ZFS Storage 7120 is an enterprise class two socket rackmount x64 system powered by the Intel Xeon processor It packs high performance and room for growth with four PCIe slots and 18 DIMM slots into a compact 2U footprint Refer to http www oracle com us products servers storage storage nas overview index html http www oracle com us products servers storage storage nas overview index html for the most recent component specification Refer to the Implementing Fibre Channel SAN Boot with Oracle s Sun ZFS Storage Appliance whitepaper at http www oracle com technetwork articles servers storage admin fbsanboot 365291 html http www oracle com technetwork articles servers storage admin fbsanboot 365291 htm1 for details on FC SAN boot solutions using the Sun ZFS Storage 7120 The 7120 is a standalone controller that consists of an internal SAS 2 HBA providing disk shelf expansion write flash acceleration and 11 x 300GB 15K 600GB 15K 1TB 7 2K 2TB 7 2K or 3TB 7 2K hard drive storage The SAS 2 storage fabric s
107. eneral memory fault go to the Maintenance gt Hardware screen of the BUI and click on the details icon on the controller Then click DIMMs to locate the faulted component indicated by the warningicon To identify a specific memory module that has faulted you must open the storage controller and use the amber status LEDs on the motherboard Caution This procedure requires that you handle components that are sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components 1 Power off the storage controller 2 Extend the storage controller from the rack 3 Remove the top cover and air filter Chapter 2 Hardware Maintenance 61 253 2 4 Disconnectthe cables attached to the drive backplane and the cable attached to the USB board 5 Fully loosen the four screws securing the drive cage to the chassis 1 6 Raisethe drive cage to the vertical position 2 62 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 2 7 Fully loosen the thumbscrews that secure the air duct 8 Slide the left L thumbscrew bracket forward and off of the mounting pins 9 Slide the right R thumbscrew bracket rearward and off the mounting pins Set the thumbscrew bracket aside 10 Lift the air duct straight up and off the alignment pins Set the air duct aside Chapter 2 Hardware
108. eplace disk shelf chassis components Connecting to Attached Storage cabling storage controllers to the disk shelves Hardware Faults Connect to ILOM to diagnose hardware faults System view system disks manage support bundles Updates manage appliance software Configuration Backup backup and restore appliance configuration Problems view current problems Logs view appliance logs Workflows manage and execute workflows ZS3 2 Hardware Overview This section describes the internal and external components of the Oracle ZFS Storage ZS3 2 controller Use this information when preparing to service replaceable components Refer to the following topics for procedural instructions Controller Tasks replace system controller components Disk Shelf Tasks replace disk shelf components Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 Controller Overview The ZS3 2 controller is an enterprise class rackmount x64 system powered by the Intel Xeon processor It provides high performance and room for growth with expandable PCIe slots and 16 DIMM slots in a compact 2U footprint CPU Memory Readzilla Boot Drive PCle Slots HBA 2x8 core 256GB 1 4L 6TB 2x2 5 inch 2dedicated 4 4 port 4x4 2 1GHz 16x16GB SAS 2 SSDs 900GB open PCIe SAS 2 6Gb s SAS 2 Options external Refer to http www oracle com us products servers storage storage nas overview index html http w
109. ere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch To remove the top cover 84 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 Simultaneously lift both cover latches in an upward motion Lift up and remove the top cover 6 Locate the PCle card position in the storage controller see Single and Cluster Controller Configurations 7 To replace the PCIe card Disengage the PCle card slot crossbar from its locked position and rotate the crossbar into an upright position Remove the retaining screw that holds the PCIe card to the chassis Chapter 2 Hardware Maintenance 85 Z53 4 86 Carefully remove the PCIe card from the connector and clean the slot with filtered compressed air if necessary Install the replacement PCIe card into the PCIe card slot Install the retaining screw to hold the PCIe card to the chassis Return the crossbar to its closed and locked position To install the top cover Place the top cover on the chassis 1 so that it is forward ofthe rear ofthe storage controller by about an inch 2 5 cm Slide the top cover toward the rear of the chassis 2 until it seats Press down on the cover with both hands until both latches engage
110. ersion ak SUNW iwashi_plus 2013 06 05 0 0 1 1 6 Copyright c 2008 2013 Oracle and or its affiliates All rights reserved dorab console login Hardware Firmware Updates Following the application ofa software upgrade any hardware for which the upgrade includes newer versions of firmware will be upgraded There are several types of devices for which firmware upgrades may be made available each has distinct characteristics Disks storage enclosures and certain internal SAS devices will be upgraded in the background When this is occurring the firmware upgrade progress will be displayed in the left panel ofthe Maintenance System BUI view or in the maintenance system updates CLI context These firmware updates are almost always hardware related though it may briefly show some number of outstanding updates when applying certain deferred updates to components other than hardware As of 2010Q3 4 when there are outstanding updates an informational or warning icon will appear next to the number of updates remaining Clicking the icon brings up the Firmware Updates dialog which lists the current remaining updates For each update we also show the current version ofthe component the time ofthe last attempted update as well as the reason why the last attempt didn t succeed Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates We consider any outstanding updates to be in one of 3 states Pending In Pro
111. gress and Failed An update begins in the Pending state and is periodically retried at which time it moves into the In Progress state If we fail to upgrade due to a transient condition the upgrade is moved back to the Pending state and otherwise to the Failed state In general it is only an indication ofa problem if There are updates in the Failed state Updates remain in the Pending state or in limbo between the Pending and In Progress states for an extended period of time more than half an hour without the number of remaining updates decreasing The following conditions don t indicate a problem Disks firmware updates are shown as pending for extended periods of time with a status message indicating that they are not part of any pool This is expected given that we only update disk firmware for disks that are part of a pool In order to update these disks add them to a pool There are multiple chassis being updated we are making progress the number of remaining updates decreases and some of the chassis transiently appear pending with a status indicating that some disk has only one path This is also expected since when we update a chassis we may reset one of its expanders Resetting an expander causes some disks to temporarily have only one path and as a result upgrades to other chassis will be held back until it is safe to do so again non disruptively Note that currently the Firmware Updates dialog doesn t a
112. h Accelerator F20 card is a FRU and must be replaced by an Oracle service representative All HBAs must be of the same type Ensure that you upgrade your system software before installing a newly released HBA Chapter2 Hardware Maintenance 133 7x20 134 You must shut down the controller before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one ofthe following methods Login to the BUL and click the power icon on the left side of the masthead SSH into the storage system and issue the maintenance system poweroff command SSH or serial console into the service processor SP and issue the stop SYS command Use a pen or non conducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown wherein all applications and files will be closed abruptly without saving press and hold the power button for at least four seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller is in standby power mode Disconnect the AC power cords from the rear panel of the storage controller Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches While squeezing the slide release latches slowly pull the
113. h is in the O off position Plug the power cord into the new power supply and attach the power cord tie strap to the power cord Place the power supply on off switch to the I on position The Power OK status LED should be a steady green and all other indicators should be off Go to the Maintenance gt Hardware section of the BUI and click the right arrow icon Q atthe beginning ofthe appropriate disk shelf row As appropriate for the failure click either PSU or Fan to verify that the icon is green for the newly installed power supply with fan module Replacing an I O Module The following disk shelves have I O Modules IOMs a Oracle Storage Drive Enclosure DE2 24P Oracle Storage Drive Enclosure DE2 24C The I O Modules IOMs which are similar to SIM boards are hot swappable so you can replace them without removing power to the system A faulted I O Module is indicated by an amber LED Go to the Maintenance gt Hardware section of the BUI click the right arrow icon at the beginning of the appropriate disk shelf row and then click Slot to view details or click the locate icon X to turn on the locator LED Important Do not remove a component if you do not have an immediate replacement The disk shelf must not be operated without all components in place Locate the failed I O Module at the back of the disk shelf Label and disconnect the I O Module interface cables Using your thumb and forefinger squeeze the rel
114. hat the status icon is green for the newly installed power supply Memory To identify a specific memory module that has faulted you must open the storage controller and use the amber status LEDs on the motherboard To identify a general memory fault go to the Maintenance gt Hardware screen of the BUI and click on the details icon on the controller Then click DIMMs to locate the faulted component indicated by the warning icon v Caution This procedure requires that you handle components that are sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components Chapter 2 Hardware Maintenance 127 7x20 You must shut down the appliance before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one ofthe following methods 6 Log in to the BUI and click the power icon on the left side of the masthead SSH into the appliance and issue the maintenance system poweroff command SSH or serial console into the service processor SP and issue the stop SYS command Use a pen or nonconducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown during which all applications and files will be closed abruptly without saving press and hold the power button for at least fo
115. he battery if the storage controller fails to maintain the proper time when powered off and disconnected from the network You will need a small No 1 flat blade non metallic screwdriver or equivalent You must shut down the appliance before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one ofthe following methods Login to the BUI and click the power icon on the left side of the masthead SSH into the storage system and issue the maintenance system poweroff command SSH or serial console into the service processor and issue the stop SYS command Use a pen or non conducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown wherein all applications and files will be closed abruptly without saving press and hold the power button for at least four seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller isin standby power mode Disconnect the AC power cords from the rear panel of the storage controller Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch 7120 or 7320
116. i iie eq etri esiti 14 Hardware View ua avessi tiva o beret AEAEE DOR ED CUERO TREE DO EON tae 15 BUD 22 atc ise se eter o D DAE o redde rcu bee a x RN DARREN 15 ep e EAE EET E E S E EA TES N M S 21 UC Tee 23 Hardware Maintenance sese tnter ente t tn tn intrent R 27 Ein ED M 27 T ttoductiOn 2 nnd vt ed nb tei e ea enn eva i Re ann 27 VU teamed ZS3 2 Hardware Overview Controller OVeEVIOW idit rd vit edes ires na reci vitii uelis FR FR C reU Physical Specifications DD M 33 Electrical Specifications sssssssssssnsssssssecsssncsecsnssecassassoensssssseseesesssseessenensenessescensesecsensensesenseneess 33 AcousticNoise EMISSIONS lt i odere ib iere ei e weit aee eri e eiut 34 Internal Components anita dire n od D OO eo dd P hoec qi inde 34 Motherboard Memory and PCIe Cards eene tnnt 36 Optional Cable Management Arm 40 Attached Storage ueccteeie cde dn a dudeqaere ene dede dedidit eiiis 41 VA 41 Z534 Hardware Overview aii oxaevo die teteie Qro BR T ON RR DRE rH d 41 Controller Overviews inniinn a n a R R E S E RRR 41 Contents ZS3 2 CRU Maintenance Procedures Prepare the Controller for Servicing Internal Components sss 53 Controller Replacement Tasks uoti EDI peto ed dee e a i ai rr e desde 56 Oracle DE2 and Sun Disk Shelf Inte
117. iance Related Documentation Refer to the following documentation for installation instructions hardware overviews service procedures and software update notes Installation Guide Analytics Guide and Administration Guide http www oracle com technetwork documentation Third Party Web Site References Third party URLs are referenced in this document and provide additional related information Note Oracle is not responsible for the availability of third party Web sites mentioned in this document Oracle does not endorse and is not responsible or liable for any content advertising products or other materials that are available on or through such sites or resources Oracle will not be responsible or liable for any actual or alleged damage or loss caused by or in connection with the use of or reliance on any such content goods or services that are available on or through such sites or resources Preface Access to Oracle Support Oracle customers have access to electronic support through My Oracle Support For information visit http www oracle com pls topic lookup ctx acc amp id info or visit http www oracle com pls topic lookup ctx acc amp id trs if you are hearing impaired 10 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 CHAPTER 1 Introduction Overview Introduction The Oracle ZFS Storage Appliance family of products provides efficient file and block data
118. ich you just updated If you do nothing this entry will boot by default completing the update The previous entries are rollback targets that can be used to initiate a rollback to previous versions of the system software Rollback is discussed below GNU GRUB version 0 97 612K lower 2087424K upper memory Sun ZFS Storage 7120 2013 06 05 0 0 1 1 6 Sun ZFS Storage 7120 2011 04 2 25151 2 Use the and v keys to select which entry is highlighted Press enter to boot the selected OS e to edit the commands before booting or c for a command line As the system boots up using the new system software you will see some special messages on the first boot indicating that an update is completing and noting the previous and new versions ofthe system software SunOS Release 5 11 Version ak generic 2013 06 05 0 0 1 1 6 64 bit Copyright c 1983 2013 Oracle and or its affiliates All rights reserved System update in progress Updating from ak nas 2011 04 2 8 Updating to ak nas 2013 06 0 4 4 2 1 1 2 5 0 0 1 1 6 Cloning active datasets done Upgrading var ak home 16 blocks Upgrading etc svc profile 176 blocks Upgrading var apache2 4432 blocks Upgrading var sadm 5040 blocks Upgrading var svc 0 blocks Upgrading var dhcp duid done Upgrading var pkg 208800 blocks Upgrading var ak logadm conf done Adjusting system dump and system cores done Upgrading var crypto pkcsll conf
119. iled component Go to the Maintenance gt Hardware screen Click the right arrow icon on the Storage System or Disk Shelf which has the fault icon Locate the fault icon in the lists of hardware components and click it The image should be updated to show where that component is physically located Optionally click the locate icon X for that component if the component has it The LED on the component will begin to flash Chapter 1 Introduction 23 Hardware 24 CLI To turn on the locate LED using the CLI run the following commands Go to the maintenance hardware context hostname maintenance hardware List the appliance components hostname maintenance hardware list NAME STATE MODEL SERIAL chassis 000 hostname ok Sun Storage 7320 unknown chassis 001 000000000C faulted 34410 000000000C Select the chassis and list its components hostname maintenance hardware select chassis 001 hostname maintenance chassis 001 list disk fan psu slot Select the component type and show all available disks hostname maintenance chassis Q01 gt select disk hostname maintenance chassis 001 disk show Disks LABEL STATE MANUFACTURER MODEL SERIAL disk 000 HDD 0 ok ST3500630NS ST3500630NS 9QG1ACNJ disk 001 HDD 1 faulted ST3500630NS ST3500630NS 90G1A77R disk 002 HDD 2 ok ST3500630NS X ST3500630NS 90G1AC3Z disk 003 HDD3 ok ST3500630NS X ST3500630NS 9QG1ACKW disk 004 HDD 4 ok ST3500630NS ST3500630NS 9QG1ACKF d
120. iliarity with the Oracle ZFS Storage Appliance clustering model if you are not familiar with the clustering concepts and terminology please read about clustering concepts in the System Administration Guide first To describe the rolling upgrade procedure this document will refer to the two clustered storage controllers as A and B where A is the controller that will be updated first and B is the controller that will be updated second A key best practice in rolling upgrades is that each controller should be upgraded at a time when it is not providing service to clients The procedure described here meets this requirement In addition all general upgrade best practices described above also apply to rolling upgrades Important Do not perform a takeover operation while an upgrade is in progress 1 Use the CLI or BUI to upload the software update image to both storage controllers 2 Select which controller to update first Ifa controller has no storage pools update that controller first In the following steps controller A is updated first so if any clients are using controller A s storage pool s they will experience a takeover induced availability delay first 3 Log in to controller A and use the CLI maintenance system reboot command or the BUI power icon in the masthead and select the reboot option to reboot controller A Controller B takes over resources from controller A 4 Log in to controller A and use the CLI or BUI to apply the
121. indicators and the Locator status indicator Locator button are dim 11 Push the release tabs on the side of each rail and slowly slide the storage controller into the rack Power Supply Storage controllers are equipped with redundant hot swappable power supplies Ifa power supply fails and you do not have a replacement leave the failed power supply installed to ensure proper air flow A faulted power supply is indicated by an amber colored status LED 1 Gain access to the rear of the storage controller where the faulted power supply is located 2 Ifacable management arm CMA is installed press and hold the CMA release tab and rotate the arm out ofthe way 3 Disconnect the power cord from the faulted power supply 4 Remove the power supply Chapter2 Hardware Maintenance 77 Z53 4 5 Grasp the power supply handle and press the release latch to remove the power supply 6 Align thereplacement power supply with the empty power supply chassis bay 7 Slidethe power supply into the bay until it is fully seated The following figure shows the power supply 8 Connectthe power cord to the power supply 9 Verify that the green AC Present status indicator is lit 10 Close the CMA inserting the CMA into the rear left rail bracket 11 Go to the Maintenance Hardware screen of the BUI Click amp details icon for the controller and then click power supply to verify that the status icon is green for the newly
122. installed power supply Memory To identify a specific memory module that has faulted you must open the storage controller and use the amber status LEDs on the motherboard To identify a general memory fault go to the Maintenance Hardware screen of the BUI and click on the details icon amp on the controller Then click DIMMs to locate the faulted component indicated by the warning icon wv Caution This procedure requires that you handle components that are sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components 78 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 You must shut down the appliance before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one of the following methods Log in to the BUI and click the power icon on the left side of the masthead SSH into the appliance and issue the maintenance system poweroff command SSH or serial console into the service processor SP and issue the stop SYS command Use a pen or nonconducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown during which all applications and files will be closed abruptly without saving press and hold the power button for at least f
123. ion shows the Fault remind button on the 7120 The following illustration shows the Fault remind button on the 7420 Chapter2 Hardware Maintenance 129 7x20 8 7420 Identify the memory riser that hosts the faulted DIMM by the Service Required status indicator Lift the memory riser straight up to remove it from the motherboard and place it onan antistatic mat 9 Rotate both DIMM slot ejectors outward as far as they will go and carefully lift the faulted DIMM straight up to remove it from the socket 10 Line up the replacement DIMM with the connector aligning the notch with the key to ensure that the component is oriented correctly 11 Push the DIMM into the connector until the ejector tabs lock the component in place 12 7120 or 7320 Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 To replace the cover Place the top cover on the chassis so that it hangs over the rear ofthe storage controller by about an inch 2 5 cm 130 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 Slide the top cover forward until it seats Close the fan cover and engage the fan cover latches The cover must be completely closed for the storage controller to power on 13 7420 To replace the cover Push the memory riser module into the associated CPU memory riser slot until the rise
124. is incompatible with older software releases Once the update is applied rolling back to previous versions will result in undefined behavior For these updates you will always be given an explicit option of applying them automatically during upgrade or applying them after the fact They are therefore referred to as deferred updates Chapter 3 System Maintenance 201 Updates 202 When applying an update to a version with incompatible version changes you will be given an option to apply these version changes as part of the upgrade For each version change the benefits ofapplying the change will be presented to you The default is to not apply them requiring you to return to the updates view and apply them once the system has rebooted after the upgrade is applied This allows you to verify that the rest ofthe software is functional and a rollback is not required before applying the update If you elect to not apply deferred updates during an upgrade you can return to the updates view at any point to apply the update If deferred updates are available for the current software version they will appear as a list below the current set of available updates with an Apply button to apply the updates Deferred updates in a cluster take effect on both storage controllers simultaneously and can only be applied while both controllers are operational Because deferred updates are listed only for resources present on the local storage controller i
125. is new setting Applying this update is equivalent to upgrading the on disk ZFS pool to ZFS Pool Version Summary User Quotas 214 User Quotas Deferred Update With the 2009 Q3 software release the system now supports user and group quotas on a per share basis In order to make use of this feature a deferred update must be applied to upgrade all shares in the system to support this feature Applying this deferred update also allows the current usage user or group to be queried on a per filesystem or per project basis To quote the product documentation Quotas can be set on a user or group at the filesystem level These enforce physical data usage based on the POSIX or Windows identity of the owner or group of the file or directory There are some significant differences between user and group quotas and filesystem and project data quotas Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Dedup COMSTAR Be sure to read the complete documentation under Space Management before attempting to use user or group quotas This update is applied in the background and takes time proportional to the number of shares and amount of data on the system Until this deferred update is finished attempt to apply user quotas will produce an error indicating that the update is still in progress COMSTAR Deferred Update The COMSTAR framework relies on a ZFS pool upgrade for complete support of persistent group reserv
126. isk 005 HDD 5 ok ST3500630NS ST3500630NS 9QG1ACPM disk 006 HDD 6 ok ST3500630NS ST3500630NS 9QG1ACRR disk 007 HDD 7 ok ST3500630NS ST3500630NS 9QG1ACGD disk 008 HDD 8 ok ST3500630NS ST3500630NS 9QG1ACG4 disk 009 HDD 9 ok ST3500630NS ST3500630NS 9QG1ABDZ disk 010 HDD 10 ok ST3500630NS X ST3500630NS 90G1A769 disk 011 HDD 11 ok ST3500630NS ST3500630NS 90G1AC27 disk 012 HDD 12 ok ST3500630NS ST3500630NS 9QG1AC41 disk 013 HDD 13 ok ST3500630NS ST3500630NS 90G1ACQ5 disk 014 HDD 14 ok ST3500630NS ST3500630NS 9QG1ACKA disk 015 HDD 15 ok ST3500630NS X ST3500630NS 9QG1AC5Y disk 016 HDD 16 ok ST3500630NS ST3500630NS 90G1ACQ2 disk 017 HDD 17 ok ST3500630NS ST3500630NS 90G1A76S disk 018 HDD 18 ok ST3500630NS ST3500630NS 9QG1ACDY disk 019 HDD 19 ok ST3500630NS ST3500630NS 9QG1AC3Y disk 020 HDD 20 ok ST3500630NS ST3500630NS 90G1ACG6 disk 021 HDD 21 ok ST3500630NS X ST3500630NS 9QG1AC3X disk 022 HDD 22 ok ST3500630NS ST3500630NS 9QG1ACHL disk 023 HDD 23 ok ST3500630NS ST3500630NS 9QG1ABLW Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Hardware Select the faulted disk and turn on the locate LED hostname maintenance chassis 001 disk select disk 001 hostname maintenance chassis 001 disk 001 gt set locate true locate true uncommitted hostname maintenance chassis 001 disk 001 gt commit Chapter 1 Introduction 25 26 CHAPTER 2 Hardware Maintenance
127. l amber AC OK green 10 PCIe slots 5 9 4 Power supply unit 1 AC inlet 11 Network management NET MGT port 5 System status LEDs Power green Attention amber Locate white 12 Serial management SER MGT port 6 PCIe slots 0 4 13 DB 15 video connector 7 Cluster card slot Connectors The serial management connector SER MGT is an RJ 45 connector and provides a terminal connection to the SP console The network management connector NET MGT is an RJ 45 connector and provides a LAN interface to the SP console NET MGT 112 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 There are four RJ 45 Gigabit Ethernet connectors NETO NET1 NET2 NET3 located on the motherboard that operate at 10 100 1000 Mbit sec These network interfaces must be configured before use Electrical Specifications The following list shows the electrical specifications for 7420 controllers Note The power dissipation numbers listed are the maximum rated for the power supply and are nota rating ofthe actual power consumption of the appliance Input Nominal frequencies 50 60Hz AC operating range 100 127 VAC for 2 CPUs and 200 240 VAC for 2 or 4 CPUs Maximum current AC RMS 12A 100 VAC 12A 200 VAC Power Dissipation Max power consumption 1800 W Max heat output 6143 BTU hr Volt Ampere rating 1837 VA 240 VAC 0 98 P F Internal Boards The 7420 controller chassis has
128. lds up static electricity during the removal and replacement ofa CRU Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Shelf Electrostatic Discharge Precautions Removing Power from the Disk Shelf Most disk shelf components are hot swappable you do not need to remove power when replacing them Do not remove a component if you do not have an immediate replacement The disk shelf must not be operated without all components in place Powering off or removing all SAS chains from a disk shelf will trigger a kernel panic on the controller s to prevent data loss unless the shelf is part of an NSPF no single point of failure data pool To avoid this shut down the controller s before decommissioning the shelf For details on NSPF profiles see Profile Configuration Stop all input and output to and from the disk shelf Wait approximately two minutes until all disk activity indicators have stopped flashing Place the power supply on offswitches to the O off position pe Qus ES x Disconnect the power cords from the external power source for the cabinet Note All power cords must be disconnected to completely remove power from the disk shelf Tasks Shelf Tasks Replacing a Drive The shelf drives are hot swappable and may be replaced without removing power from the shelf The replacement drive must be ofthe same capacity and type as the drive to be replaced To avoid possible data loss when
129. le Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third party content products and services Oracle Corporation and its affiliates will not be responsible for any loss costs or damages incurred due to your access to or use of third party content products or services Copyright O 2009 2014 Oracle et ou ses affili s Tous droits r serv s Ce logiciel et la documentation qui l accompagne sont prot g s par les lois sur la propri t intellectuelle Ils sont conc d s sous licence et soumis des restrictions d utilisation et de divulgation Sauf disposition de votre contrat de licence ou de laloi vous ne pouvez pas copier reproduire traduire diffuser modifier breveter transmettre distribuer exposer ex cuter publier ou afficher le logiciel m me partiellement sous quelque forme et par quelque proc d que ce soit Par ailleurs il est interdit de proc der toute ing nierie inverse du logiciel de le d sassembler ou de le d compiler except des fins d interop rabilit avec des logiciels tiers ou tel que prescrit par la loi Les informations fournies dans ce document sont susceptibles de modification sans pr avis Par ailleurs Oracle Corporation ne garantit pas quelles soient exemptes d erreurs et vous invite le cas ch ant a lui en faire part par crit Si ce logiciel ou la documentation qui l accompagne est conc d sous licence au
130. le ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 Serial Management Connector The serial management connector SER MGT is an RJ 45 port and provides a terminal connection to the service processor SP console SER MGT Network Management Connector The network management connector NET MGT is an RJ 45 port and provides an alternate terminal interface to the SP console Ethernet Ports The ZS3 2 has four RJ 45 10 Gigabit Ethernet IOGbE network connectors labeled NET 3 NET 2 NET 1 and NET 0 left to right on the rear panel The ports operate at 100 Mbits sec 1000 Mbits sec or 10 Gbits sec Use these ports to connect the appliance to the network gt 3E si i gt wes J Ner2 Jeo Neri nero 100 MbE 10 GbE The LEDs located above each NET port are Link Activity left and Speed right indicators for each port as described in this table Connection Type EEETerminology Speed LED Color Transfer Rate Fast Ethernet 100BASE TX Off 100 Mbits sec Gigabit Ethernet 1000BASE T Amber 1000 Mbits sec Chapter 2 Hardware Maintenance 31 253 2 32 Connection Type EEE Terminology Speed LED Color Transfer Rate 10 Gigabit 10GBASE T Green 10000 Mbits sec Ethernet Cluster I O Ports Two cluster serial ports 0 and 1 and one Ethernet port provide communication between two controllers to form a cluster configuration For information about how to c
131. lease tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 9 Connectthe power cords to the power supplies 10 Verify that the green AC Present status indicator is lit 11 Close the CMA inserting the CMA into the rear left rail bracket 12 Goto the Maintenance Hardware screen of the BUI Click amp details icon for the controller and then click power supply to verify that the status icon is green for the newly installed power supply Fan Modules Removing a fan module is a cold service operation You must power offthe appliance before you remove the fan module There will be a loss of access to the storage unless the system is ina clustered configuration The status of each fan module is represented by a single bi color LED The LEDs are located on the motherboard near each fan module and are visible from the rear panel by looking through the grille below the risers 1 To locate the chassis you want to service click the associated locate icon on the Maintenance Hardware screen of the BUI or issue the set SYS LOCATE status on command at the service processor SP prompt The locate LED will flash on the controller chassis 2 Identify the faulted fan module by locating the corresponding Service Required status indicator or by clicking the X locate icon on Maintenance gt Hardware screen of the BUI for the fan you want to replac
132. letely closed for the storage controller to power on 11 7420 To install the top cover Place the top cover on the chassis 1 so that it is forward ofthe rear ofthe storage controller by about an inch 2 5 cm Slide the top cover toward the rear ofthe chassis 2 until it seats Press down on the cover with both hands until both latches engage 12 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 138 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 13 14 15 16 17 18 Connect the power cords to the power supplies Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit Connect data cables to the PCIe card routing them through the cable management arm Go to the Maintenance Hardware screen of the BUI and click the details icon amp on the controller Then click Slots to verify the status ofthe new component The status indicator should appear green Install the disk shelf and connect the expansion storage Battery You might need to replace t
133. linent toute responsabilit quant aux dommages caus s par l utilisation de ce logiciel ou mat riel pour ce type d applications Oracle et Java sont des marques d pos es d Oracle Corporation et ou de ses affili s Tout autre nom mentionn peut correspondre des marques appartenant d autres propri taires qu Oracle Intel Xeon sont des marques ou des marques d pos es d Intel Corporation Toutes les marques SPARC sont utilis es sous licence et sont des marques ou des marques d pos es de SPARC International Inc AMD Opteron lelogo AMD etlelogo AMD Opteron sont des marques ou des marques d pos es d Advanced Micro Devices UNIX est une marque d pos e d The Open Group Ce logiciel ou mat riel et la documentation qui l accompagne peuvent fournir des informations ou des liens donnant acc s des contenus des produits et des services manant de tiers Oracle Corporation et ses affili s d clinent toute responsabilit ou garantie expresse quant aux contenus produits ou services manant de tiers En aucun cas Oracle Corporation et ses affili s ne sauraient tre tenus pour responsables des pertes subies des co ts occasionn s ou des dommages caus s par l acc s des contenus produits ou services tiers ou leur utilisation 140127025097 Contents Preface REIR E DL 9 Introduction DORIA NT EIER RD RECINTO D EE MVE UCI OM soha di ohad unie E E EAE OU E 11 Hardware esas asesinado desi meii ite cote ot eie dei
134. lla 232 Gigabit Ethernet Host Bus Adapter Host Channel Adapter Hard Disk Drive HyperText Transfer Protocol combines disk flash and DRAM into a single coherent and seamless data store icons visible in the BUI I O Module similar to a SIM Internet Small Computer System Interface arestricted BUI mode where a user may only view one specific screen Level 2 Adaptive Replacement Cache Lightweight Directory Access Protocol light emitting diode write IOPS accelerator Logical Unit top section of BUI screen a new screen element for a specific function Network File System Network Interface Card Network Information Service Peripheral Component Interconnect Express Power Cooling Module consisting of a PSU and one or more fans provide storage space that is shared across all filesystems and LUNs a collection of shares Power Supply Unit included with fans in a power cooling module PCM quad data rate read optimized flash SSD for the LZARC Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZFS Remote replicating shares to another appliance Replication Rollback reverts all ofthe system software and all ofthe metadata settings ofthe system back to their state just prior to applying an update SAS Serial Attached SCSI SAS 2 Serial Attached SCSI 2 0 SATA Serial ATA Schema configurable properties for shares Scripting automating CLI tasks Service appliance service software Share Z
135. lownfish maintenance system updates ak nas 2009 04 03 1 0 gt help properties Properties that are valid in this context version date status Update media version Update release date Update media status update zfs upgrade gt Apply incompatible storage pool update clownfish maintenance system updates ak nas 2009 04 03 1 0 gt get update zfs upgrade version 2009 04 03 1 0 date 2009 4 3 08 45 01 status AKUP WAITING deferred clownfish maintenance system updates ak nas 2009 04 03 1 0 gt set update zfs upgrade onreboot update zfs upgrade onreboot clownfish maintenance system updates ak nas 2009 04 03 1 0 gt 212 After you set any properties execute the upgrade command You are prompted for confirmation and assuming an affirmative the upgrade begins dory maintenance system updates select ak nas 2009 10 14 1 0 nd dory maintenance system updates ak nasQ2009 10 14 1 0 nd upgrade The selected software update requires a system reboot in order to take effect The system will automatically reboot at the end of the update process The update will take several minutes At any time during this process you can cancel the update with Control C Are you sure Y N y Updating from ak nasQ2009 10 11 1 0 Backing up smf 5 done Loading media metadata done Selecting alternate product SUNW iwashi Installing Sun Storage 7120 2009 10 14 1 0 pkg sun com ak SUNW iwashi 2009 10
136. n a cluster it may be the case that deferred updates are available only for resources active on the peer controller In a cluster it is therefore necessary to check both storage controllers to determine the availability of deferred updates Note Replication does not work across deferred updates After deferred updates are applied that increment the stream format version it is no longer possible to replicate to an older system See Incompatible target Replication Failure for a description Feature Version introduced passthrough x aclinherit property 2009 Q2 0 0 User quotas 2009 Q3 0 0 COMSTAR 2009 Q3 0 0 Triple Parity RAID 2009 Q3 0 0 Dedup 2010 Q1 0 0 Replication 2010 Q1 0 0 Received Properties 2010 Q1 0 0 Slim ZIL 2010 Q3 1 0 Snapshot Deletion 2010 Q3 1 0 Recursive Snapshots 2010 Q3 1 0 Multi Replace 2010 Q3 1 0 RAIDZ Mirror 2011 1 0 0 Optional Child Directory 2011 1 0 0 Multiple Initiator Groups per LUN 2011 1 8 0 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates Reboot After an Update Following the completion ofthe update process the system will reboot automatically If you have the serial console open you will notice during this reboot that multiple GRUB menu entries are available ordered from the newest software at the top to the oldest software at the bottom The default menu entry will be the top the new software to wh
137. n lights and remains lit The Maintenance Hardware screen of the BUI provides status ofthe replacement on the Details page for DIMMs PCle Cards Removing a PCIe card is a cold service operation You must power offthe appliance before you remove the card There will be a loss of access to the storage unless the system is in a clustered configuration Go to the Maintenance gt Hardware screen of the BUI and click the details icon on the controller and then click Slots to locate the faulted component Caution This procedure requires that you handle components that are sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components Chapter2 Hardware Maintenance 65 253 2 66 Power offthe storage controller Extend the storage controller from the rack Remove the top cover Locate the PCIe card position in the storage controller see ZS3 2 Overview Sr gm cpes coz Disconnect any data cables connected to the cards on the PCIe riser you want to replace Label the cables for proper connection later 6 Loosen the two captive screws securing the riser to the motherboard Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 7 Lifttheriser straight up and out ofits socket on the motherboard Chapter2 Hardware Maintenance 67 253 2 9 Liftthe PCIe card off ofthe br
138. nder normal operating circumstances Clicking the information icon 8 will bring up a dialog with information about each path This includes which HBAs are connected to the disk shelf and the state of any paths If the disks within the disk shelf are not currently configured as part of a storage pool complete path information will not be available though it displays two paths to the chassis Xx Toggle the locate LED for this disk shelf If the LED is currently on then this Locate 7f Baas indicator will be flashing Chassis Detail To view the chassis details click on the right arrow icon amp or one ofthe alternative forms described above This view includes some of the same controls in the upper left state name locate reset poweroff as well as listings of all the components in the chassis At the left is a set of images describing the chassis If there are multiple views then you can switch between them by clicking on the name of the view above the image For each view faulted components will be highlighted in red In addition the currently selected component will be highlighted in the image Clicking on a component within the image will select the corresponding component in the list to the right A tab is present for each component type in the following list Each component type has a state icon which will be orange if there is a faulted component of the given type m Disk Slot CPU controller only Memory
139. network management connector NET MGT is an RJ 45 connector on the motherboard and provides an alternate terminal interface to the SP console NET MGT There are four RJ 45 Gigabit Ethernet connectors NETO NET1 NET2 NET3 located on the motherboard that operate at 10 100 1000 Mbit sec These network interfaces must be configured before use Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 7420 Attached Storage The 7320 single and cluster controller configurations allow a single chain of one to six disk shelves Any combination of disk only and Logzilla capable shelves may be combined within the chain in any order The cabling configurations are unchanged Half populated shelf configurations are not supported See Also Controller Details Disk Shelf Overview Disk Shelf Maintenance Procedures 7420 Hardware Overview Use the information on this page as a preparation reference for servicing replaceable components of 7420 controllers Refer to the following for procedural instructions Controller Tasks replace system controller components Disk Shelf Tasks replace disk shelf components Appliance Overview The Sun ZFS Storage 7420 Appliance consists of a standalone storage controller or two storage controllers in a high availability cluster configuration and one to 36 disk shelves The following table describes the 7420 configuration options Note 7420 M2 controllers ca
140. ng 1837 VA 240 VAC 0 98 P F Internal Boards The ZS3 4 controller chassis contains the following field replaceable units FRUs FRUs are not customer serviceable and should only be replaced by trained Oracle service technicians Motherboard The motherboard includes CPU modules slots for eight DIMM risers memory control subsystems and the service processor SP subsystem The SP subsystem controls the host power and monitors host system events power and environmental The SP controller draws power from the host s 3 3V standby supply rail which is available whenever the system is receiving AC input power even when the system is turned off Power Distribution Board The power distribution board distributes main 12V power from the power supplies to the rest of the system It is directly connected to the Vertical PDB card and to the motherboard through a bus bar and ribbon cable It also supports a top cover interlock kill switch In the controller the power supplies connect to the power supply backplane which connects to the power distribution board ma Vertical PDB Card The vertical power distribution board or Paddle Card serves as the interconnect between the power distribution board and the fan power boards hard drive backplane and I O board Power Supply Backplane Card This board connects the power distribution board to power supplies 0 and 1 Fan Power Boards The two fan power boards are FRUs and carry p
141. ning the software from before the upgrade Important Do not perform a cluster takeover operation or a reboot while an upgrade is in progress Following the post upgrade reboot component firmware will be updated see firmware updates below which will take additional time that depends on the size of the system configuration and the amount of firmware that has changed since the previously installed version was delivered very large configurations may take several hours to complete all firmware upgrades once the update itself has been applied For details on the update process using the BUI or CLI review the sections below Preconditions Best practices include verifying several preconditions prior to applying an update Whenever possible administrators should ensure that these preconditions are satisfied immediately prior to applying an update on the storage controller In a clustered environment these should be verified on both storage controllers before applying the update to either one Ensure that any resilvering operations have completed This can be observed in Configuration Storage or the equivalent CLI context Ensure that there are no active problems Chapter 3 System Maintenance 197 Updates 198 Verify that firmware updates are not in progress m Check the most recent product release notes for additional preconditions that should be observed for the software release to which you are upgrading Update Health
142. nnot be clustered with 7420 controllers Mktg Part ProductID Number CPU DIMMs Readzilla Boot Drive Mfg Part Number 7420 M2 7107089 4x8 core 2 0GHz 8GB 16GB 1 6TB SAS 900GB SAS 7075466 7420 M2 7107090 4x10 core 8GB 16GB 1 6TB SAS 900GB SAS 7075470 2 40GHz 7420 7100566 4x8 core 2 0GHz 8GB 16GB 512GBSATA 500GBSATA 7014572 7420 7100568 4x10 core 8GB 16GB 512GBSATA 500GBSATA 7014573 2 40GHz Chapter2 Hardware Maintenance 109 7420 You can identify your configuration by looking at the product id on the BUI Maintenance screen or by using the CLI configuration version show command You can also check the physical label on the boot drive as shown in the boot drive illustration below Refer to http oracle com ZFSStorage http oracle com ZFSStorage for the most recent component specification Chassis The 3U chassis form factor dimensions are as follows Dimension Measurement Dimension Measurement Height 13 3cm 5 25in Depth 70 6 cm 27 8 in Width 43 7 cm 17 19in Weight 16 36 kg 96 lbs Front Panel gt INN QXeXSX4XSX eX 7X8 WZ 8 14 43 s YS QD 9 10 Figure Legend 1 Locator LED and button white 7 Power Supply PS Service Required LED 13 SSD 2 optional 2 Service Required LED amber 8 Over Temperature Warning LED 14 SSD 3 optional 3 Power OK LED green 9 USB 2 0 Connectors 15 SSD 4 optional 4 Power button 10 DB 1
143. not directory users Alerts and thresholds and their associated rules NOTE In a cluster configuration configuration backups are visible only on the node on which they were created or imported A configuration backup does not include User data shares and LUNs Your user data must be backed up separately using NDMP backup software snapshots and or remote replication Chapter3 System Maintenance 219 ConfigurationBackup m User passwords for directory users These remain stored solely in your separate network directory service such as LDAP or Active Directory and will not be stored in the backup or restored Metadata directly associated with user data such as snapshot schedules user quotas compression settings and other attributes of shares and LUNs Analytics and logs Events can be redirected to external SNMP trap receivers or e mail destinations using Alerts rules System software The system software is automatically backed up as part of the System Update capability Restore Impact The restore operation takes a selected configuration backup and modifies all of the corresponding system settings to reflect those in the backup including removing aspects ofthe configuration that were not present at the time of the backup Administrators should adhere to the following guidelines when planning a restore Scheduled downtime The restore process takes several minutes to complete and will impact servi
144. ns for Oracle ZFS Storage ZS3 2 7320 clustered controllers with one HBA Using mixed disk shelves on a controller requires the following The controller must use only 4X4 port SAS 2 HBAs Do not use mixed disk shelves in the same chain Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model 4X4 port SAS 2 HBAs are only supported with release AK 2013 1 0 and later Cable Diagrams left noh oss fig 2 One HBA and multiple disk shelves in two chains DE2 24 on the left 188 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 7420 with DE2 and Sun Disk Shelves fig 3 Eight disk shelves in two chains DE2 24 on the left ZS3 4 7420 with DE2 and Sun Disk Shelves Connecting ZS3 4 7420 Standalone Controllers to Mixed Disk Shelves The following figures show a subset of the supported configurations for Oracle ZFS Storage ZS3 4 7420 standalone controllers with two HBAs Using mixed disk shelves on a controller requires the following The controller must use only 4X4 port SAS 2 HBAs Do not use mixed disk shelves in the same chain Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model 4X4 port SAS 2 HBAs are only supported with release AK 2013 1 0 and later Cable Diagrams
145. ntenance 155 Shelf Shelf 156 Disk Shelf Maintenance Procedures This section provides procedural details for customer replaceable units CRUs of any disk shelf or drive enclosure that attaches to the Oracle ZFS Storage Appliance family of products Refer to Disk Shelf Overview for component specifications and diagrams Prerequisites Read the information in the overview section for your controller to become familiar with the replaceable parts of the system m 7120 7320 7420 ZS3 2 ZS3 4 Overviews component diagrams and specifications Follow the instructions in the Electrostatic Discharge Precautions section Safety Information Follow all cautions warnings and instructions marked on the equipment and described in Important Safety Information for Oracle s Hardware Systems Electrostatic Discharge Precautions Remove all plastic vinyl and foam material from the work area Wear an antistatic wrist strap at all times when handling any CRU Before handling any CRU discharge any static electricity by touching a grounded surface Do not remove a CRU from its antistatic protective bag until you are ready to install it After removing a CRU from the chassis immediately place it in an antistatic bag or antistatic packaging Handle any card that is part of a CRU by its edges only and avoid touching the components or circuitry Do not slide a CRU over any surface Limit body movement which bui
146. o indicate a ready state Chapter2 Hardware Maintenance 159 Shelf 160 8 Goto the Maintenance Hardware section of the BUI click the right arrow icon Q atthe beginning of the appropriate disk shelf row and then click Disk to verify that the diskicon is green for the newly installed disk Replacing a Power Supply Disk shelves are provided with redundant power supplies to prevent loss of service due to component failure Each power supply is accompanied by one or more chassis cooling fans in one customer replaceable unit CRU Power supplies are hot swappable meaning they can be replaced one at a time without removing power from the disk shelf The modules can produce a high energy hazard and should only be replaced by instructed individuals with authorized access to the equipment Separate indicator LEDs on the rear panel represent the operational state of power supplies and fans individually see the rear panel illustration for details Failed components are indicated by amber LEDs as well as amber icons in the administrative BUI Go to the Maintenance gt Hardware section of the BUL click the right arrow icon Q at the beginning ofthe appropriate disk shelf row then select PSU or Fan to view details on the respective components You can also click the locate icon X to flash the chassis locator LED Important Do not remove a component if you do not have an immediate replacement The disk shelf must not be operated wi
147. ommand Delete a Saved Configuration Then the destroy command deletes a saved configuration host maintenance system configs destroy cb2f005f cf2b 608f 90db fc7a0503db2a Are you sure you want to delete the saved configuration new y host maintenance system configs Export a Saved Configuration The export command exports a saved configuration by means ofexecuting an HTTP or FTP PUT operation against a remote HTTP or FTP server You can also use the export function to Chapter3 System Maintenance 223 Problems Problems 224 export the file to a share on the appliance itself that has the HTTP or FTP protocol enabled for writing You can enter a username and password for authentication to the remote server if one is required Import a Saved Configuration The import command imports a saved configuration by means of executing an HTTP or FTP GET operation against a remote HTTP or FTP server You can also use the import function to import a configuration stored in a share on the appliance itself that has the HTTP or FTP protocol enabled for reading You can enter a username and password for authentication to the remote server if one is required Problems To aid serviceability the appliance detects persistent hardware failures faults and software failures defects often included under faults and reports them as active problems on this screen If the phone home service is enabled active problems are automatically rep
148. onnect cables to form a cluster see the Cluster configuration topic Removing RJ 45 Cables from Cluster Serial Ports CAUTION When disconnecting an RJ 45 cable from a cluster serial port 0 and 1 use extreme care not to damage the internal RJ 45 receptacle To properly remove an RJ 45 cable froma cluster serial port do the following 1 Using your index finger press down fully on the RJ 45 release tab Be sure the tab is fully disengaged from the port T 7 S 85 77 2 1 v DA Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 2 Using your thumb and middle finger apply a slight downward pressure while pulling the below the plug Physical Specifications The ZS3 2 controller 2U chassis dimensions are as follows Dimension Measurement Dimension Measurement Height 3 44in 87 4mm Depth 20 25 in 514 mm Width 17 52in 445 mm Weight 41 231b 18 70 kg Electrical Specifications The ZS3 2 controller electrical specifications are listed below The power dissipation numbers listed are the maximum rated power numbers for the power supply The numbers are not a rating ofthe actual power consumption ofthe appliance Connectors TwoCl3connectors which work on 110 220v outlets Input Nominal frequencies 50 60Hz 47 to 63 Hz range Nominal voltage range 100 to 120 200 to 240 VAC Maximum current AC RMS 6 8 A 1004120 V 3 4 A 2003240 V AC ope
149. onsole E The network management connector NET MGT is an RJ 45 connector and provides a LAN interface to the SP console E NET MGT There are four RJ 45 Gigabit Ethernet connectors NETO NET1 NET2 NET3 located on the motherboard that operate at 10 100 1000 Mbit sec These network interfaces must be configured before use PCle Options This table describes the PCle base configuration and optional slot assignments for ZS3 4 standalone and cluster configurations Slot Mfg Part Number Description Max Note 0 7047852 8 port 6Gb s SAS 2 internal 1 Base configuration HBA Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 Slot Mfg Part Number Description Max Note 1 7067091 4 port 4x4 SAS 2 6Gb s 4 Base configuration SAS 2 external HBA 2 7067091 4 port 4x4 SAS 2 6Gb s 4 Additional optional back end SAS 2 external HBA 2 7070006 2 port 1OGbE copper NIC 4 Optional recommended front end 2 375 3696 01 2 port Infiniband CX2 HCA 4 Optional recommended front end 2 7070195 4 port IGbE copper NIC 4 Optional recommended UTP front end 2 7051223 2 port IOGbE optical NIC 6 Optional recommended front end 2 371 4325 02 2 port 8Gb FC HBA 4 Optional FC target or initiator backup 3 7070006 2 port 1OGbE copper NIC 4 Optional recommended front end 3 375 3696 01 2 portInfiniband CXK2 HCA 4 Optional recommended front end 3 7070195 4 port IGbE copper
150. operating system integrated software any programs installed on the hardware and or documentation shall be subject to license terms and license restrictions applicable to the programs No other rights are granted to the U S Government This software or hardware is developed for general use in a variety of information management applications It is not developed or intended for use in any inherently dangerous applications including applications that may create a risk of personal injury If you use this software or hardware in dangerous applications then you shall be responsible to take all appropriate fail safe backup redundancy and other measures to ensure its safe use Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications Oracle and Java are registered trademarks of Oracle and or its affiliates Other names may be trademarks of their respective owners Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International Inc AMD Opteron the AMD logo and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices UNIX is a registered trademark of The Open Group This software or hardware and documentation may provide access to or information on content products and services from third parties Orac
151. or in the 7120 and 7320 storage controllers Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 Leaving the door open for more than 60 seconds while the storage controller is running might cause it to overheat and shut down 7420 The following illustration shows the fan modules in the 7420 storage controller Chapter 2 Hardware Maintenance 123 7x20 310 I mure n my BORG 1 1 1 To locate the chassis you want to service click the associated 3 locate icon on the Maintenance gt Hardware screen of the BUI or issue the set SYS LOCATE status on command at the service processor SP prompt The locate LED will flash on the controller chassis 2 Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches 4 While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch 5 7120 or 7320 To replace the fan module Open the fan module door while unlatching the release tabs on the door Identify the faulted fan module by locating the corresponding Service Required status indicator or by clicking the 3 locate icon on Maintenance gt Hardware screen of the BUI for the fan you want to replace Using thumb and forefinger pull the fan module up
152. orted to Oracle Support where a support case may be opened depending on the service contract and the nature of the fault Active problems display For each problem the appliance reports what happened when the problem was detected the severity and type of the problem and whether it has been phoned home Below are some example faults as they would be displayed in the BUI Date Description Type Phoned Home 2009 09 16 SMART health monitoring firmware reported thata disk Major Fault Never 13 56 36 failure is imminent 2009 09 05 A disk of a different type cache log or data was inserted Minor Never 17 42 55 into a slot The newly inserted device must be ofthe same Fault type 2009 08 21 The ZFS pool has experienced currently unrecoverable Major Never 16 40 37 1 O failures Error 2009 07 16 A memory module is experiencing excessive correctable Major Fault Never 22 03 22 errors affecting large numbers of pages Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Problems This information can also be viewed in the CLI gefilte maintenance problems show Problems COMPONENT DIAGNOSED TYPE DESCRIPTION problem 000 2010 7 27 00 02 49 Major Fault SMART health monitoring firmware reported that a failure is imminent on disk HDD 17 Selecting any fault shows more information about the fault including the impact to the system affected components the system s automated response if an
153. ot change any ofthe current settings unless explicitly requested User data on the storage pool including projects and shares will not be affected To perform an initial setup m BUTI click the INITIAL SETUP button on the Maintenance gt System screen CLI enter the maintenance system context then enter the setup command Chapter3 System Maintenance 195 Updates Updates 196 Factory Reset Factory reset will reset the appliance configuration back to factory settings of the current software version and reboot the appliance All configuration changes will be lost and the appliance will need to go through initial configuration again as when it was first installed User data on the storage pool including projects and shares will not be affected however the pool will need to be imported as part of the initial setup process To perform a factory reset BUT click the FACTORY RESET button on the Maintenance gt System screen CLI enter the maintenance system context then issue the factoryreset command GRUB Add c to the GRUB menu on the line beginning with kernel Note Factory reset of a single controller while configured into a cluster is not supported The controller must be unclustered first System Updates The system update feature provides customers developers and field personnel with the ability to update a system s software after the system is installed Software updates are delivered as op
154. our 1 86GHz or 2 00GHz CPUs with two memory risers required by each CPU Four or eight 4GB or 8GB DDR3 DIMMs are installed on each riser accommodating up to 256GB of memory for two CPUs or up to 512GB for four CPUs M M 4 f Taal Refer to the service label on the cover for DIMM placement information On every memory riser slots DO D2 D4 and D6 must be populated optionally slots D1 D3 D5 and D7 may be populated as a group on all installed memory risers All DIMMs in the system must be identical Chapter 2 Hardware Maintenance 115 7420 116 D4 O_aa B D6 Jis 2 DE 4 MBO 4 a Ds 2 ms D2 3 Se D1 D gt i DO MB1 DIMM names in appliance logs and the Maintenance Hardware view are displayed with the full name such as SYS MB P0 D7 Fan Modules The Fan Modules and Fan Module LEDs ofthe storage controller are shown in the following figure The following LEDs are lit when a fan module fault is detected HIE t ES Jie 310 3 m Frontand rear Service Action Required LEDs Fan Module Service Action Required TOP LED on the front of the server Fan Fault LED on or adjacent to the faulty fan module The system Overtemp LED might light if a fan fault causes an increase in system operating temperature Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7420 NIC HBA Options This table describes NIC HBA PCIe c
155. our seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller is in standby power mode Disconnect the AC power cords from the rear panel of the storage controller Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch To remove the top cover Chapter 2 Hardware Maintenance 79 Z53 4 Simultaneously lift both cover latches in an upward motion Lift up and remove the top cover 6 To locate the DIMM you want to service press the Fault Remind Button on the storage controller The following illustration shows the Fault remind button on the ZS3 4 80 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 7 Identify the memory riser that hosts the faulted DIMM by the Service Required status indicator Lift the memory riser straight up to remove it from the motherboard and place it on an antistatic mat 8 Rotate both DIMM slot ejectors outward as far as they will go and carefully lift the faulted DIMM straight up to remove it from the socket 9 Line up the replacement DIMM with the connector aligning the notch with the key to ensure that the component is oriented correctl
156. ove update media highlight the corresponding row and click on the TI trash icon Applying Deferred Updates Any deferred updates will be displayed below the list of available updates If no deferred updates are available no list will be displayed The deferred updates will describe what effects they will Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates have on the system Clicking the Apply button will apply all available deferred updates Deferred updates will apply to both nodes in a cluster and the cluster peer must be up and available to apply any deferred updates Updating via the CLI Because you log into the appliance to use the CLL the upload as described above is actually a download To download the media onto the appliance via the CLI execute the download command in maintenance system updates dory maintenance system updates download dory maintenance system updates download uncommitted get url unset user unset password unset You must set the url property to be a valid URL for the download This may be either local to your network or over the internet The URL can be either HTTP beginning with http or FTP beginning with ftp If user authentication is required it may be a part of the URL e g ftp myusername mypasswd myserver export foo or you may leave the username and password out of the URL and instead set the user and password properties
157. ower to the controller fan modules In addition they contain fan module status LEDs and transfer I2C data for the fan modules Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 Drive Backplane The six drive backplane includes the connectors for the drives as well as the interconnect for the I O board Power and Locator buttons and system component status LEDs Each drive has an LED indicator for Power Activity Fault and Locate Components The components of the ZS3 4 controller are shown in the following figure and identified in the legend Figure Legend 1 Motherboard 7 CPUs and heatsinks 2 Low profile PCIe cards 8 Memory risers 3 Power supplies 9 Fan board 4 Power supply backplane 10 Fan modules 5 Drive backplane 11 Boot drives and SSDs 6 System lithium battery Chapter 2 Hardware Maintenance 45 Z53 4 CPU and Memory The ZS3 4 controller supports 16GB DDR3 DIMMs installed on all eight risers accommodating 1 TB of memory ERN LRA Refer to the service label on the cover for DIMM placement information On every memory riser slots D0 D2 D4 and D6 must be populated optionally slots D1 D3 D5 and D7 may be populated as a group on all installed memory risers All DIMMs in the system must be identical pi Y 0 1 s Er 3 nens D6 Rr 2 7 O 4 DERE D3 2 ga 02 s Di C nE Do MB1 46 Oracle
158. r module locks in place Place the top cover on the chassis so that it is forward ofthe rear ofthe storage controller by about an inch 2 5 cm Chapter2 Hardware Maintenance 131 7x20 Slide the top cover toward the rear of the chassis until it seats and press down on the cover with both hands until both latches engage 132 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 The following image shows the 7420 chassis 15 Connect the power cords to the power supplies 16 Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in 17 Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit The Maintenance Hardware screen of the BUI provides status ofthe replacement on the Details page for DIMMs PCle Cards and Risers Go to the Maintenance gt Hardware screen of the BUI and click the details icon on the controller and then click Slots to locate the faulted component Caution This procedure requires that you handle components that are sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components Note that the 7120 Sun Flas
159. r 7 Universal power input connector 3 DC power status indicator 8 Right ejection arm and captive screw latch 4 Power supply status indicator 5 Power on off switch Disk Shelf Configurations The following tables describe and provide part numbers for the supported expansion storage shelves Oracle Storage Drive Enclosure DE2 24P Mktg Part Number Description 7103910 Drive Enclosure DE2 24P Base Chassis Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Mktg Part Number Description 7103911 300GB 10Krpm SAS 2 2 5 HDD 7103912 900GB 10Krpm SAS 2 2 5 HDD 7103915 73GB SSD SAS 2 2 5 Write Flash Accelerator 7103917 Filler Panel Drive Enclosure DE2 24P Oracle Storage Drive Enclosure DE2 24C Mktg Part Number Description 7103914 Drive Enclosure DE2 24C Base Chassis 7103913 3TB 72Krpm SAS 2 3 5 HDD 7103916 73GB SSD XATO SAS 2 2 5 2 5 to 3 5 Drive Adapter 7103918 Filler Panel Drive Enclosure DE2 24C Sun Disk Shelf DS2 Mktg Part Number Description DS2 0BASE Sun Disk Shelf DS2 24x3 5 SAS 2 DS2 HD2T 2TB 7 2Krpm SAS 2 3 5 HDD 7101765 3TB 7 2Krpm SAS 2 3 5 HDD 7101274 300GB 15Krpm SAS 2 3 5 HDD 7101276 600GB 15Krpm SAS 2 3 5 HDD 7101197 73GB SSD XATO 3 5 DS2 LOGFILLER Sun Disk Shelf DS2 24x3 5 LOGFiller DS2 4URK 19U Disk Shelf Rail Kit Chapter 2 Hardware Mai
160. r storage expansion The CPUs are Intel Xeon 5600 series 2 40GHz 80W with 4 core processors The standard memory configuration is 96GB 6 x 8GB DDR3 1333 low voltage LV DIMMs per CPU Memory can be upgraded to 144GB using 9 x 8GB DDR3 1333 LV DIMMs per CPU for a total of 18 x 8GB for two CPUs Earlier versions of the 7320 controller included 24GB base 48GB or 72GB memory options The clustered configuration simply uses two servers and a cluster card in each server for a heartbeat connection between the servers All user accessible storage is provided by one to six disk shelves that are external to the server s RAID functions are managed by software Solid state 18GB SAS 1 drives 7320 SAS 2 are used for a high performance write cache known as LogZilla or ZFS intent log ZIL devices and are used in place of up to four of the 24 drives in a disk shelf The remaining 20 drives are available for storage Refer to the http www oracle com us products servers storage storage unified storage index html http www oracle com us products servers storage storage unified storage index htm1 for the most recent component specification The 7320 is a SAS 2 Serial Attached SCSI 2 0 device that consists of an HBA disk shelf and disks 1TB and 2TB SAS 2 The SAS 2 storage fabric supports a greater number of targets greater bandwidth higher reliability and bigger scale than the SAS 1 fabric Boards The 7320 storage controller cha
161. r your controller to become familiar with the replaceable parts of the system 7120 7320 7420 Follow the instructions in the Safety Information and Required Tools and Information sections Safety Information This section contains safety information that you must follow when servicing the storage system For your protection observe the following safety precautions when setting up your equipment Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 Do not remove the side panels or run the storage system with the side panels removed Hazardous voltage is present that could cause injury The covers and panels must be in place for proper air flow to prevent equipment damage Follow all cautions warnings and instructions marked on the equipment and described in Important Safety Information for Sun Hardware Systems Ensure that the voltage and frequency of your power source match the voltage inscribed on the electrical rating label Follow the electrostatic discharge safety practices Electrostatic discharge ESD sensitive devices such as PCI cards HDDs SSDs and memory cards require special handling Circuit boards and HDDs contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards Do not touch the components without using antistatic precautions
162. rating range 90 264 VAC Output 3 3 VDCSTBY 3 0A m 12 VDC 86 7A Power Dissipation Chapter 2 Hardware Maintenance 33 253 2 34 Max power consumption 890 W max Max heat output 3026 BTU hr Volt Ampere rating 908 VA 240 VAC 0 98P F Acoustic Noise Emissions In compliance with the requirements defined in ISO 7779 the workplace dependent noise level of this product is less than 70 db A Internal Components The ZS3 2 chassis has the following field replaceable units FRUs FRUs are not customer serviceable and must be replaced by trained Oracle service technicians PCIe Risers There are three risers per system each attached to the rear of the motherboard Each riser supports two PCIe cards which are customer replaceable units CRUs Motherboard The motherboard includes CPU modules slots for 16 DIMMs memory control subsystems and the service processor SP subsystem The SP subsystem controls the host power and monitors host system events power and environmental The SP controller draws power from the host 3 3V standby supply rail which is available whenever the system is receiving AC input power even when the system is turned off Power Distribution Board The power distribution board distributes main 12V power from the power supplies to the rest of the controller It is directly connected to the connector break out board and to the motherboard through a bus bar and ribbon cable It al
163. redundant hot swappable power supplies If a power supply fails and you do not have a replacement leave the failed power supply installed to ensure proper air flow A faulted power supply is indicated by an amber colored status LED 1 Gain access to the rear ofthe storage controller where the faulted power supply is located 2 Ifa cable management arm CMA is installed press and hold the CMA release tab and rotate the arm out ofthe way 3 Disconnect the power cord from the faulted power supply 4 Remove the power supply 7120 or 7320 Release the latch then remove the power supply Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 amp 126 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 A 7x20 7420 Grasp the power supply handle and press the release latch to remove the power supply 5 Align thereplacement power supply with the empty power supply chassis bay 6 Slide the power supply into the bay until it is fully seated The following figure shows the 7420 power supply 7 Connect the power cord to the power supply 8 Verify that the green AC Present status indicator is lit 9 Close the CMA inserting the CMA into the rear left rail bracket 10 Go to the Maintenance gt Hardware screen of the BUI Click amp details icon for the controller and then click power supply to verify t
164. resson the release button to slide the drive into the chassis until the release lever moves slightly inward 8 Press the release lever fully closed with a click securing the drive into the drive slot The Oracle ZFS Storage system software automatically detects and configures the new drive The device appears in the BUI Maintenance Hardware screen when you view details for the controller or drive shelf Power Supplies Storage controllers are equipped with redundant hot swappable power supplies Ifa power supply fails and you do not have a replacement leave the failed power supply installed to ensure proper air flow A faulted power supply is indicated by an amber colored status LED 1 Gain access to the rear of the storage controller where the faulted power supply is located Chapter2 Hardware Maintenance 57 253 2 2 Ifa cable management arm CMA is installed press and hold the CMA release tab and rotate the arm out of the way 3 Disconnect the power cord from the faulted power supply 4 Pressthe release tab to the left and pull on the handle 5 Continue to pull on the handle to slide the power supply out ofthe chassis 6 Align the replacement power supply with the empty power supply chassis bay 58 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 7 Slidethe power supply into the bay until the power supply seats and the release tab clicks 8 Push the re
165. rial console as usual and during boot before the ten second timeout use the arrow key to move the menu selection down to one of the earlier entries GNU GRUB version 0 97 612K lower 2087424K upper memory Use the and v keys to select which entry is highlighted Press enter to boot the selected OS e to edit the commands before booting or c for a command line After the rollback boot menu entry is selected the system will boot the old kernel software but the rollback must be manually confirmed on the console in order to commit the rollback which will effectively remove all changes to the system that have happened since as described above The confirmation step looks like this SunOS Release 5 11 Version ak generic 2011 04 24 4 2 1 1 28 64 bit Copyright c 1983 2010 Oracle and or its affiliates All rights reserved System rollback in progress Rollback to ak nasQ2011 04 24 4 2 1 1 28 Proceed with system rollback y n mow Entering y proceeds with the rollback and the system will complete boot using the prior snapshot Entering n cancels the rollback and immediately reboots the system allowing the administrator to select a different boot image i e the current system software or an older snapshot Chapter 3 System Maintenance 207 Updates 208 Cluster Upgrade Ina clustered system a rolling upgrade can be performed eliminating downtime while the upgrade is performed This section assumes fam
166. rnal sensors indicate that a fanisno Minor Fault 19 20 15 longer operating correctly Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Logs Date Event ID Description Type 2009 8 21 c91c7b32 83ce 6da8 e51e a553964bbdbcThe ZFS pool has experienced currently Major Error 16 40 48 unrecoverable I O failures These faults will generate alert log entries and so will use the alert reporting settings such as sending email if configured Faults that require administrator attention will appear in Problems System This is the operating system log available to read via the appliance interfaces This may be useful when troubleshooting complex issues but should only be checked after first examining the alert and fault logs The following are example system log entries as they would appear in the BUI Time Module Priority Description 2009 10 1114 13 38 ntpdate error no server suitable for synchronization found 2009 10 11 14 03 52 genunix notice MSunOS Release 5 11 Version ak generic 2009 10 10 1 0 64 bit 2009 10 1114 02 04 genunix notice done 2009 10 1114 02 01 genunix notice syncing file systems 2009 10 1113 52 16 nxge warning WARNING nxge gt nxge rxdma databuf free DDI Audit The audit log records user activity events including login and logout to the BUI and CLI and administrative actions If session annotations are used see Users each audit entry should
167. roperability ovp 73 Satety IntoEmiationo acie iie ageoetioeteitiseradenatras tci sile r a A 73 Required Tools and Information eerte ttes 73 Chassis Serial Number Controller Replacement Tasks Oracle DE2 and Sun Disk Shelf Interoperability sse 89 7120 Hardware Overview Chassis Overview Electrical Specifications Internal Components xc coiere edo idit vede a edi de eer eiie eco ra 93 Standalone Controller Configurations sssssee eene 98 Attached Storage 7320 Replaceable Components ite uitinire d ndo D nd ndo ERE ads 105 7320 Single and Cluster Controller Configurations seen 107 Appliance OVERVIEW sca dd entente nente eine e dede Abre e diet 109 Gur I EE 110 Internal Boards cianic t e ert pie ride tede i D o thee a i kr e ea t dn 113 COInDpODCBtss dads otdiivarcnteindsidur setae bas ipu cseteris 114 Attached SLOPQBE ai a en A REDE NT Ep ID ceed OR AT RIS 120 T EOE TEREE EITE T EEEE EE EENEI E 120 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Contents 7x20 CRU Maintenance Procedures eee rennen tne ntes tenni 120 PHOT EQUISILCS Safety INfOrMaAtiOM e Required Tools and Information Chassis Serial Number iced anre P RUNE EAE RE UNEERE NENN
168. rsive Snapshots Deferred Update Mult Replace sisicsivcscspe cuacieniacdelasa serani siaii Saia hati A SEEE S AEEA SS Multi Replace Deferred Upd te e re dateien trit metet ret ug RAIDZ MEO 5 5 50 ORE URRDHONURER D OEU EE sess saaestscyeas sussptusasisesewaneiaes RAIDZ Mirror Deferred Update eee tentent ttt tentent Optional Child Dir Introduction Multiple Initiator Groups per LUN seseeeeeeeentteen tentent tentent tenente ii oduc ON o eae eren OUR UR ROO POUND CE Conhp rationBacktip od oett i Idea AS Configuration Backup idee dienen oum iiton orandi inde Re UR eee Problems 2d ede tei da Ein see tec dad redet s e d i CE e edd i de ee adde a don ene d aud Problems Active problems display rete er iaiia ain Oe aN ENE Repairing problems atacan ean ean epe in n Aen n Aa AT BLA Related features sie ieoiterentiedetiede eene died ove have eda re dni eee iei Preface The Oracle ZFS Storage Appliance Customer Service Manual contains hardware overviews and maintenance procedures for the Oracle ZFS Storage Appliance This documentation is also available while using the Browser User Interface accessible via the Help button The appliance documentation may be updated using the System Upgrade procedure documented in the System Maintenance chapter of this book Who Should Use This Book These notes are for users and system administrators who service and use the Oracle ZFS Storage Appl
169. select audit show Entries ENTRY TIME SUMMARY entry 000 2009 10 15 00 59 37 root console Enabled datalink igb0 service entry 001 2009 10 15 00 59 39 root lt console gt Enabled interface igb0 service Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Logs entry 002 2009 10 15 01 00 39 root console User logged in entry 003 2009 10 15 01 41 44 root console Enabled nis service entry 004 2009 10 15 01 42 01 root console Imported storage pool pool 0 entry 005 2009 10 15 17 56 30 root console User logged in entry 006 2009 10 15 17 56 53 root deimos sf fishworks com User logged in via CLI entry 007 2009 10 15 18 00 21 root deimos sf fishworks com User logged out of CLI entry 008 2009 10 15 18 14 47 root console Browser session timed out entry 009 2009 10 15 20 46 27 root deimos sf fishworks com User logged in via CLI entry 010 2009 10 15 21 51 46 root console Rebooted appliance entry 011 2009 10 15 21 51 46 root console User logged out entry 012 2009 10 15 21 56 44 root deimos sf fishworks com User logged in via CLI Most recent entries are displayed at the bottom ofthe list Entry details Alllog entry details are available when selecting that entry and running show caji maintenance logs select audit caji maintenance logs audit select entry 000 show Properties timestamp 2009 10 15 00 59 37 user root address console summary
170. select chassis 000 chassis 000 gt select cpu cpu select cpu 000 cpu 000 show CPU 0 true false AMD Quad Core AMD Opteron tm Processor 8356 1002 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Hardware revision 03 cores 4 speed 2 14G Restarting the Appliance Only issue the restart command as instructed by Oracle Service personnel This function is only available via the CLI and is not the same as a reboot power cycle via the BUI The restart command is a software only operation that restarts the management server which could impact some client services like replication During execution both the CLI and BUI are not available wait for the appliance to return to normal operation Under the maintenance system context issue the command restart tarpon maintenance system gt restart Performing a Diagnostic Reboot Only issue the diagnostic reboot command as instructed by Oracle Service personnel A diagnostic reboot gathers diagnostic information before power cycling the appliance This operation could take a long time to complete and could cause adverse results if not performed properly Do not reboot when system affecting operations are occurring such as upgrading firmware executing commands and configuring or unconfiguring storage Under the maintenance system context issue the command diagreboot tarpon maintenance system gt diagreboot Tasks BUI Locating a fa
171. services to clients over a network and a rich set of data services that can be applied to the data stored on the system Controllers ZS3 2 ZS3 4 7120 7320 7420 Expansion Storage Disk Shelves Overview 12 Protocols Oracle ZFS Storage appliances include support for a variety of industry standard client protocols including SMB NFS HTTP and HTTPS WebDAV iSCSI FC SRP iSER FTP a SFTP Key Features Oracle ZFS Storage systems also include new technologies to deliver the best storage price performance and unprecedented observability of your workloads in production including Analytics a system for dynamically observing the behavior of your system in real time and viewing data graphically The ZFS Hybrid Storage Pool composed of optional Flash memory devices for acceleration of reads and writes low power high capacity disks and DRAM memory all managed transparently as a single data hierarchy Data Services To manage the data that you export using these protocols you can configure your Oracle ZFS Storage system using the built in collection of advanced data services including LICENSE NOTICE Remote Replication and Cloning may be evaluated free of charge but each feature requires that an independent license be purchased separately for use in production After the evaluation period these features must either be licensed or deactivated Oracle reserves the right to audit for licensing complian
172. six siii xxiii xiii Cur ur opm sr ugr opm gm opm gm opm gm uj gm gm gm gm gr gm gm ug gr ur ug ug Up to four Logzilla SSDs are supported per disk shelf Logzilla SSDs should be populated in order of slots 20 21 22 and 23 Oracle Storage Drive Enclosure DE2 24C and Sun Disk Shelf 218 Xp Up to four Logzilla SSDs are supported per disk shelf Logzilla SSDs should be populated in order of slots 20 21 22 and 23 The Oracle Storage Drive Enclosure DE2 24C is shown and represents both models Chapter2 Hardware Maintenance 145 Shelf 146 Front Panel Indicators The following figures show the front panel indicators Oracle Storage Drive Enclosure DE2 24P 9 ta La ta kat WU JJ 1 LL LL ES EID EIC EIC EC EIC ESIO EC EXC EIC Figure Legend 1 System power indicator 4 Drive fault indicator 2 Module fault indicator 5 Power Activity indicator 3 Locate indicator Oracle Storage Drive Enclosure DE2 24C Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf XXX XXX Lo e e Figure Legend 1 System power indicator 4 Power Activity indicator 2 Module fault indicator 5 Drive fault indicator 3 Locate indicator Sun Disk Shelf Chapter2 Hardware Maintenance 147 Shelf Figure Legend 1 Locate button and indicator 4 Disk ready to be removed indicator 7 Over temperature warning indicator 2 Sy
173. so supports a top cover interlock kill switch The power supplies connect directly to the power distribution board Storage Drive Backplane The storage drive backplane includes the connectors for the storage drives as well as the interconnect for the I O board power and locator buttons and system component status LEDs The system has an 8 disk backplane Each drive has an LED indicator for Power Activity Fault and Locate Storage Power and Fan Components The ZS3 2 controller s internal storage power and cooling components are described in the following figure and legend A component identified as a field replaceable unit FRU must be replaced by trained Oracle service technicians Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 253 2 Figure Legend 1Drives 5 Drive power cable FRU 9 Fan modules 2 Air Filter 6 Drive signal cable FRU 10Fantray 3 Chassis FRU 7 Top cover 11 USB board FRU 4LED board FRU 8 PDB signal cable FRU 12 Power supplies Internal Cables The ZS3 2 controller contains the following field replaceable unit FRU internal cables FRUs are not customer serviceable and must be replaced by trained Oracle service technicians Chapter 2 Hardware Maintenance 35 253 2 Figure Legend 1 PDB signal cable 3 Drive signal cable 5 Drive power cable 2 USB board cable 4LED board cable Motherboard Memory and PCle Cards The Z
174. software update to controller A At the end of the upgrade controller A reboots again and runs the new software version Important Do not perform an upgrade on a controller while it is providing service 5 Log in to controller B and use the CLI maintenance system reboot command or the BUI power icon in the masthead and select the reboot option to reboot controller B Controller A takes over all resources and provides service using the new software version 6 Validate the new software version on controller A and ensure that all services work correctly on the client systems 7 lfaserious problem appears roll back controller A Controller A reboots and controller B takes over and runs the previous software version When controller A recovers it will also run the previous software version 8 Ifno serious problem appears log in to controller B and use the CLI or BUI to apply the software update to controller B Controller B reboots and runs the new software version 9 Verify that all firmware updates complete Important Controller firmware updates cannot proceed if the controllers are running different system software versions 10 To restore normal operation and return resources to their respective assigned controllers log in to controller A and use the CLI or BUI to perform a failback on controller A The following table describes the state of the cluster after each step of the previous procedure Oracle ZFS Storage Appliance Cu
175. ssis has the following boards installed Note Field replaceable units FRUs should only be replaced by trained Oracle service technicians m PCIe Risers The storage controller contains three PCIe risers that are customer replaceable units CRUs and are attached to the rear of the motherboard Each riser supports one PCIe card Motherboard The motherboard is a FRU and includes CPU modules slots for 18 DIMMs memory control subsystems and the service processor SP subsystem The SP subsystem controls the host power and monitors host system events power and environmental The SP controller draws power from the host 3 3V standby supply rail which is available whenever the system is receiving AC input power even when the appliance is turned off Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7320 Power Distribution Board The power distribution board is a FRU and distributes main 12V power from the power supplies to the rest of the storage controller It is directly connected to the paddle board and to the motherboard through a bus bar and ribbon cable It also supports a top cover interlock kill switch Paddle Board The paddle board is a FRU and serves as the interconnect between the power distribution board and the fan power boards hard drive backplane and I O board Fan Board The fan boards are FRUs and carry power to the storage controller fan modules In addition they contain f
176. stem fault indicator 5 Disk fault indicator 8 SIM board fault indicator 3 System power indicator 6 Disk activity indicator 9 Power supply fault indicator Oracle Storage Drive Enclosure DE2 24P Figure Legend 1 Power Supply with Fan Module0 31 O Module 0 21 O Module 1 4 Power Supply with Fan Module 1 148 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Oracle Storage Drive Enclosure DE2 24C Figure Legend 1 Power Supply Filler Panel Slot0 4 Power Supply Filler Panel Slot3 7 I O Module Filler Panel 2 Power Supply with Fan Module1 5 I O Module Filler Panel 8 I O Module 1 3 Power Supply with Fan Module2 6 1 O Module 0 Note It is especially important that power supplies and their filler panels are in the correct slots Sun Disk Shelf Chapter 2 Hardware Maintenance 149 Shelf Figure Legend 1 Power supply modules with built in fans Power supply 0 is on the left and power supply 1 is on the right 2 Removable SAS Interface Module SIM Boards SIM 0 is on the left and SIM 1 is on the right 1 O Module Indicators The following disk shelves have I O Modules IOMs Oracle Storage Drive Enclosure DE2 24P Oracle Storage Drive Enclosure DE2 24C 150 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Figure Legend 1Fault Locate indicator 4SAS 2 Port 1 7 For Oracle
177. stomer Service Manual January 2014 E48492 02 Updates Step Controller AState Controller AVersion Controller B State Controller B Version 1 2 CLUSTERED V CLUSTERED V i STRIPPED V OWNER V 4 STRIPPED V 1 OWNER V 5 6 7 OWNER V 1 STRIPPED V 89 OWNER V 1 STRIPPED V 1 10 CLUSTERED V 1 CLUSTERED V 1 Do not make configuration changes to either storage controller while an upgrade is in progress While controllers are running different software versions configuration changes made to one controller will not be propagated to its peer controller Accessing the BUI or logging into the CLI while the controllers are running different software versions generates a warning that your configuration changes will not be propagated You can configure the appliance to generate alerts when the cluster controllers are running different software versions events Cluster rejoin mismatch and Cluster rejoin mismatch on peer Ifyou change the root password during an upgrade and then rollback the cluster the nodes will not be able to re join after the rollback Updating via the BUI Click the add icon next to Available Updates and specify the pathname on your desktop or local client of the update media During the upload a progress bar is displayed indicating the progress ofthe upload Note that on some older browsers the progress bar may not be updated continuously during the upload if you see a watch cursor just wait a minute in th
178. t during appliance operation The following are example alert log entries as they would appear in the BUI Time Event ID Description Type 2009 9 16 f18bbad1 8084 4cab c950 82ef5b8228ea An I O path from slot PCIe0 Major alert 13 01 56 to chassis JBOD 1 has been removed 2009 9 16 8fb8688c 08f2 c994 a6a5 ac6e755e53bb A disk has been inserted into Minor alert 13 01 51 slot HDD 4 of chassis JBOD 1 2009 9 16 446654 fc b898 6da5 e87e 8d23ff12d5d0 A disk has been inserted into Minor alert 13 01 51 slot HDD 15 of chassis JBOD 1 Aninfo icon amp next to the Event ID means that extended information is available Click the icon and this information will be displayed below the list of alerts The appliance can also be configured to send email raise an SNMP trap or perform other actions when particular alerts occur This is configured in the Alerts section All alerts appear in this log regardless of whether they have actions configured for them Faults The fault log records hardware and software faults This is a useful reference when troubleshooting hardware failure as timestamps are available for these hardware fault events The following are example fault log entries as they would appear in the BUI Date Event ID Description Type 2009 9 5 9e46fcOb b1a4 4e69 f10f e7dbe80794fe The device HDD 6 has failed or could Major Fault 17 42 35 not be opened 2009 9 3 d37cb5cd 88a8 6408 e82d c05576c52279Exte
179. t possible to offline a disk that would prevent access to data i e offlining both halves ofa mirror Ifthe device is an active hot spare this will also give the option of detaching the hot spare completely Once a hot spare is detached it cannot be activated except through another fault or hotplug event Online Online the disk Reverses the above operation Infiniband Host Controller Adapters Infiniband Host Controller Adapters HCA report additional properties for the list of available ports Action Description State When active the active porticon is displayed Other valid port states down init and arm are denoted by the inactive port icon 8 Mousing over the port icon will display the current port state in the tip pop up GUID The hardware assigned port GUID Speed The current port speed enabled Single Data Rate SDR Dual Data Rate DDR or Quad Data Rate QDR Service Processor The service processor behaves differently from other component nodes Instead of providing a list of components it presents a set of network properties that can be configured from the storage appliance The following properties control the behavior ofthe service processor network management port Property Description MAC Address Hardware MAC address This is read only IP Address Either DHCP or Static Controls whether DHCP should be used on the interface Source IP Address IPv4 Address
180. te has been applied the system will be able to create snapshots but will do so using the old much slower algorithm Applying this update is equivalent to upgrading the on disk ZFS pool to version 27 Chapter3 System Maintenance 217 Multi Replace Multi Replace Multi Replace Deferred Update This deferred update allows importing a pool with a missing log device and corrects the behavior ofthe system when a device that is being resilvered is itself removed or replaced These fixes rely on a ZFS pool upgrade provided by this update Before this update has been applied the system will be unable to import pools with missing log devices and will not correctly handle replacement of resilvering devices see CR 6782540 Applying this update is equivalent to upgrading the on disk ZFS pool to version 28 RAIDZ Mirror RAIDZ Mirror Deferred Update This deferred update improves both latency and throughput on several important workloads These improvements rely on a ZFS pool upgrade provided by this update Applying this update is equivalent to upgrading the on disk ZFS pool to version 29 Optional Child Dir 218 Introduction This deferred update improves list retrieval performance and replication deletion performance by improving dataset rename speed These improvements rely on a ZFS pool upgrade provided by this update Before this update has been applied the system will be able to retrieve lists and delete replications but will
181. ted by the time the management interfaces become available Chapter 3 System Maintenance 205 Updates 206 Upgrading disk or flash device firmware requires that the device be taken offline during the process If there is insufficient redundancy in the containing storage pool to allow this operation the firmware upgrade will not complete and may appear stalled Disks and flash devices that are part ofa storage pool which is currently in use by the cluster peer if any will not be upgraded Finally disks and flash devices that are not part of any storage pool will not be upgraded Upgrading the firmware in a disk shelf requires that both back end storage paths be active to all disks within all enclosures and for storage to be configured on all shelves to be upgraded For clusters with at least one active pool on each controller these restrictions mean that disk shelf firmware upgrade can be performed only by a controller that is in the owner state During the firmware upgrade process hardware may appear to be removed and inserted or offlined and onlined While alerts attributed to these actions are suppressed if you are viewing the Maintenance Hardware screen or the Configuration Storage screen you may see the effects of these upgrades in the UI in the form of missing or offline devices This is nota cause for concern however ifa device remains offline or missing for an extended period of time several minutes or more even
182. tered Next Steps After connecting attached storage apply power and configure the appliance Powering on Initial configuration Storage expansion may require adding or replacing HBAs in your configuration For information see the following sections ZS3 2 Maintenance Procedures PCIe Cards 7ZS3 4 Maintenance Procedures PCIe Cards and Risers 7x20 Maintenance Procedures PCIe Cards and Risers Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 DE2 to ZS3 2 7120 7320 Standalone Summary of Controller Configurations Maximum Controller Configurations The following table shows the maximum supported controller configurations NOTE Controllers cannot use 2X4 port SAS 2 HBAs and 4X4 port SAS 2 HBAs at the same time To use DE2 and Sun Disk Shelves together the controller must use 4X4 port SAS 2 HBAs which are only supported with release AK 2013 1 0 and later Controller Max Shelves Max 2X4port SAS 2HBA Max 4X4 port SAS 2 HBA ZS3 2 8 NA 2 ZS3 4 36 NA 4 7120 2 1 NA 7320 6 1 1 7420 36 6 6 DE2 to ZS3 2 7120 7320 Standalone Connecting ZS3 2 7120 7320 Standalone Controllers to Disk Shelves The following figures show a subset ofthe supported configurations for Oracle ZFS Storage ZS3 2 7120 7320 standalone controllers with one or two HBAs Note For hardware port locations see the PCIe Options section in the Hardware Maintenance Overview for your controller model
183. th the positive side facing upward 9 To install the top cover Place the top cover on the chassis 1 so that it is forward ofthe rear ofthe storage controller by about an inch 2 5 cm Slide the top cover toward the rear of the chassis 2 until it seats 88 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 Press down on the cover with both hands until both latches engage 10 Push the release tabs on the side of each rail and slowly push the storage controller into the rack making sure no cables obstruct the path of the controller 11 Connect the power cords to the power supplies 12 Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in 13 Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit 14 Connect data cables to the PCIe card routing them through the cable management arm 15 When the system has finished booting log in and set the time using the steps in the BUI Clock task Oracle DE2 and Sun Disk Shelf Interoperability Oracle DE2s and Sun Disk Shelves can be used together with standalone and clustered 7x20 controllers after certain hardware modifications Contact Oracle Service to update your controllers for disk sh
184. the following boards installed Field replaceable units FRUs should only be replaced by trained Oracle service technicians Motherboard The motherboard is a FRU and includes CPU modules slots for eight DIMM risers memory control subsystems and the service processor SP subsystem The SP subsystem controls the host power and monitors host system events power and environmental The SP controller draws power from the host s 3 3V standby supply rail which is available whenever the system is receiving AC input power even when the system is turned off Power Distribution Board The power distribution board is a FRU and distributes main 12V power from the power supplies to the rest ofthe system It is directly connected to the Vertical PDB card and to the motherboard through a bus bar and ribbon cable It also supports a top cover interlock kill switch In the storage controller the power supplies connect to the power supply backplane which connects to the power distribution board Vertical PDB Card The vertical power distribution board or Paddle Card is a FRU and serves as the interconnect between the power distribution board and the fan power boards hard drive backplane and I O board Power Supply Backplane Card This board connects the power distribution board to power supplies 0 and 1 Chapter2 Hardware Maintenance 113 7420 Fan Power Boards The two fan power boards are FRUs and carry power to the storage
185. the right arrow icon at the beginning of the appropriate disk shelf row and then click Slot to view details or click the locate icon 2 to turn on the locator LED Important Do not remove a component if you do not have an immediate replacement The disk shelf must not be operated without all components in place Locate the failed SIM at the back of the disk tray Label and disconnect the tray interface cables Use two hands to disconnect the SAS cable Grasp the metal body of the connector with one hand and firmly grasp and pull the tab gently toward the connector body with the other then pull the connector body outward to extract it from the bulkhead Do not twist or pull the tabin any direction other than parallel with the connector body or it may break If the tab breaks use a small sharp object such as a fine tipped screwdriver to lift the metal spring at the top of the connector shell to unlatch it Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf 4 Loosen the two extraction arm captive screws using your thumb and forefinger If the captive screws are too tight to loosen by hand use a No 2 Phillips screwdriver to loosen each screw 5 Pull each ejector tab outward and push to the sides to release and partially eject the SIM from the chassis 6 Grasp the middle of the SIM board and slide it out of the slot Chapter2 Hardware Maintenance 165 Shelf 7 10 11 12 1
186. this time which would cause an upgrade to this media to be aborted Troubleshooting Update Health Check Failures Prior to the actual update health checks are performed automatically when an update is started If an update health check fails it can cause an update to abort see Example 1 Update health checks only validate issues that can impact updates Chapter 3 System Maintenance 199 Updates l OK j System is not in an upgradeable state prerequisite healthcheck reports problems See alert log for more dorab maintenance system updates ak nas 2013 06 05 0 0 1 1 6 gt upgrade This procedure will consume several minutes and requires a system reboot upon successful update but can be aborted with Control C at any time prior to reboot A health check will validate system readiness before an update is attempted and may also be executed independently using the check command Are you sure Y N error System is not in an upgradeable state prerequisite healthcheck reports problems See alert log for more Example 1 BUI and CLI update health check failures Actions to Take to Resolve Health Check Alerts After an update health check failure you can review the Alert Log and take action to resolve each failure based on the message in the log The following table lists the update health check failures that can block an update and describes the associated Alert Log message and recommended order of steps you can take to resol
187. thout all components in place Locate the chassis and module containing the failed component Ensure the power supply on off switch is in the O off position Disconnect the power cord tie strap from the power cord and unplug the power cord from the power supply Release the lever ejection arms Oracle Storage Drive Enclosure DE2 24P or DE2 24C Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf Grasp the latch and the opposite side of the module and squeeze together to release the lever Sun Disk Shelf Using your thumb and forefinger unscrew both ejection arm captive screws until loose and swing the ejection arms out until they are fully open 5 Pull the module out of the chassis being careful not to damage the connector pins in the back Chapter2 Hardware Maintenance 161 Shelf 6 Withthe lever ejection arms fully open slide the new module into the chassis slot until it contacts the chassis backplane and the lever ejection arms begin to engage 7 Close the lever ejection arms Oracle Storage Drive Enclosure DE2 24P or DE2 24C Push the lever fully closed until you hear or feel a click Sun Disk Shelf Push the ejection arms fully closed and secure both captive screws to seat and secure the module in the chassis 162 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Shelf 10 11 Ensure the power supply on off switc
188. un Disk Shelves 179 7320 Cluster Cabling patte tet eed ter aeii aa ia 180 Connecting the 7320 Clustered Controller to Sun Disk Shelves 180 7420 Cabling 2 basse nt rec in pen sed Dea de i Ci red e a canis it D een De edd ois 181 Contents Connecting the 7420 Standalone Controller to Sun Disk Shelves 2 and 6 HBAs 181 7420 Chuster Cabling qe 184 Connecting the 7420 Clustered Controllers to Disk Shelves 2 and 6 HBAS 184 ZS3 2 7120 7320 with DE2 and Sun Disk Shelves seen 187 Connecting ZS3 2 7120 7320 Standalone Controllers to Mixed Disk Shelves 187 ZS3 2 7320 Clustered with DE2 and Sun Disk Shelves see 188 Connecting ZS3 2 7320 Storage Clustered Controllers to Mixed Disk Shelves 188 ZS3 4 7420 with DE2 and Sun Disk Shelves minnan Nr uaaa Nanis 189 Connecting ZS3 4 7420 Standalone Controllers to Mixed Disk Shelves 189 ZS3 4 7420 Clustered with DE2 and Sun Disk Shelves see 190 Connecting ZS3 4 7420 Clustered Controllers to Mixed Disk Shelves 190 SystemiDISKS ine ette similes E dono Nit tige nera te Deae EROR Ing e a o e Need ERe 193 Support Bundles ti iode teni Pape etre io nt ad ien ERR 194 Initial Setup M
189. upports a greater number of targets greater bandwidth higher reliability and bigger scale The 2U chassis form factor dimensions are as follows Dimension Measurement Dimension Measurement Height 87 6mm 3 45in Depth 765 25 mm 30 13 in Width 436 8 mm 17 2in Weight 29 54 kg 65 Ib Front Panel The following figure and legend show the front panel and the drive locations The Logzilla 3 5 SSD belongs in slot 3 and is not supported in controllers configured with the internal Sun Aura flash HBA Logzilla Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7120 4 OOC 8a 9 10 1712 13 ADSS Figure Legend 1 Locator LED button white 7HDD2 13 HDD 8 2 Service Action Required LED amber 8 HDD or SSD3 14 HDD 9 3 Power button 9 HDD 4 15 HDD 10 4 Power OK LED green 10 HDD 5 16 HDD 11 5 HDDO 11 HDD 6 17 Drive map 6HDD 1 12 HDD7 Rear Panel The following figure and legend show the rear panel Note Optional Sun Dual Port 40Gb sec 4x Infiniband QDR HCAdapter PCIe cards 375 3606 01 may be located in slots 1 2 or 3 375 3606 01 HCA expansion cards are not supported in the 10Gb network configurations Figure Legend 1 Power Supply Unit 1 6 PCIe 4 11 Network Management port 2 Power Supply Unit0 7 Boot HDD 1 12 Gbit Ethernet ports NET 0 1 2 3 3 PCIe 0 8 Boot HDD 0 13 USB 2 0 ports 0 1 Chapter 2 Hardware Maintenance 91 7120 92
190. ur seconds until the Power OK status indicator on the front panel flashes indicating that the storage controller is in standby power mode Disconnect the AC power cords from the rear panel of the storage controller Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack From the front of the storage controller release the two slide release latches While squeezing the slide release latches slowly pull the storage controller forward until the slide rails latch 7120 or 7320 Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 To remove the top cover Unlatch the fan module door pull the two release tabs back rotate the fan door to the open position and hold it there Press the top cover release button and slide the top cover to the rear about a half inch 1 3 cm Lift up and remove the top cover Also remove the air baffle by pressing the air baffle connectors outward and lifting the air baffle up and out of the server 7420 To remove the top cover 128 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 Simultaneously lift both cover latches in an upward motion Lift up and remove the top cover 7 To locate the DIMM you want to service press the Fault Remind Button on the storage controller The following illustrat
191. urations The 7320 cluster base configuration is 96GB RAM 2x2 4GHz Quad Core processors one external SAS HBA and four 10 100 1000 Ethernet ports and a Cluster card The Sun Storage 7420C Cluster Upgrade Kit XOPT 594 4680 01 contains two cluster cards with cables for converting two 7320 or two 7420 controllers to a cluster Chapter2 Hardware Maintenance 107 7320 108 The following options are available for clustered storage controllers Note When you cluster a 7320 you must identically configure the cards in both of the clustered storage controllers and you must identically configure all optional NIC HBA cards used in the clustered storage controllers in both chassis Slot Type Part Number Vendor Part Description Note 0 PCIe 375 3617 01 Intel Niantic DP Optical 10GE NIC Optional Recommended Front end 0 PCIe 375 3696 01 Mellanox InfiniBand HCA Optional Recommended Front end 0 PCle 375 3606 03 MHJH29 XTC InfiniBand HCA Optional Recommended Front end 0 PCIe 375 3481 01 Intel EXPI9404PT QP Copper NIC Optional Recommended Front end 0 PCle 371 4325 01 QLogic 8Gb DP FC HBA Optional FC Target or Initiator Backup 1 PCle 542 0298 01 Sun Fishworks Cluster Card2 Cluster Base Configuration 2 PCIe 375 3609 03 Sun Thebe External SAS HBA Cluster Base Configuration 7320 Connector Pinouts The serial management connector SERIAL MGT is an RJ 45 connector and is a terminal connection to the SP console The
192. ust be connected to the Internet either directly or through the web proxy configured on the Phone Home service screen If the upload fails the system will make another attempt After a support bundle has been successfully uploaded the support bundle and core files are automatically deleted from the system Managing Support Bundles Using the BUI To generate a support bundle click the icon next to Support Bundles on the Maintenance gt System screen You are presented with the randomly generated filename for the support bundle Provide this filename to support personnel so that they can retrieve your support bundle For each support bundle currently being generated or uploaded or which has failed to upload the following options may be available Icon Description Cancel the current operation If the bundle is being generated it will be deleted If the bundle is being uploaded the upload will be cancelled and the appliance will not retry itlater amp Download the support bundle 45 Tryagainto upload the bundle to support TIT Cancel any pending operation and delete the support bundle Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 System Managing Support Bundles Using the CLI To generate and upload a new support bundle use the sendbundle command loader maintenance system loader maintenance system sendbundle A support bundle is being created and sent
193. utomatically refresh so you would have to close it and re open it to get an updated view Applying hardware updates is always done in a completely safe manner This means that the system may be in a state where hardware updates cannot be applied This is particularly important in the context of clustered configurations During takeover and failback operations any in progress firmware upgrade will be completed pending firmware upgrades will be suspended until the takeover or failback has completed at which time the restrictions described below will be reevaluated in the context of the new cluster state and if possible firmware upgrades will resume Important Unless absolutely necessary takeover and failback operations should not be performed while firmware upgrades are in progress The rolling upgrade procedure documented below meets all of these best practices and addresses the per device class restrictions described below It should always be followed when performing upgrades in a clustered environment In both clustered and standalone environments these criteria will also be reevaluated upon any reboot or diagnostic system software restart which may cause previously suspended or incomplete firmware upgrades to resume Components internal to the storage controller such as HBAs and network devices other than disks and certain SAS devices will generally be upgraded automatically during boot these upgrades are not visible and will have comple
194. ve the issue For component faults follow the instructions for removal and installation found in the Maintenance procedures for your controller ID and Alert Log Message Failure Resolution Steps B1 System software update cannot proceed Slot label in SIM cannot be detected 1 2 4 disk shelf name is reported as absent B2 System software update cannot proceed Slot label in SIM is faulted 1 2 4 disk shelf name is faulted C1 System software update cannot proceed Some slots of SIM is missing firmware 14 disk shelf name have no firmware revision information revision information C2 System software update cannot proceed The slots of SIMs report different part 2 4 disk shelf name have non uniform part numbers numbers 200 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 Updates ID and Alert Log Message Failure Resolution Steps C5 System software update cannot proceed The slots of SIMs report different 4 disk shelf lt name gt have mixed firmware revisions lt rev1 gt firmware revisions and lt rev2 gt E1 System software update cannot proceed Disk shelf Disk shelf does nothavetwo 1 2 4 lt name gt has just one path or zero paths gt paths E2 System software update cannot proceed Disk shelf Diskshelfpathisnotonline 1 2 4 lt name gt path pathname is lt state gt E3 System software update cannot proceed Pool
195. while systems are actively clustered The clustering process means that settings are being synchronized between cluster peers and each peer appliance also is maintaining private settings For this reason you must first use the Unconfiguring Clustering procedure to un cluster the two systems Then restore the configuration backup on a selected head and then re cluster the two systems at which point the other system will automatically synchronize itself with the restored configuration 220 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ConfigurationBackup Root privileges required Configuration backups include all system metadata and therefore require all possible privileges and authorizations to create or apply Therefore unlike other delegated administrative options only the root user is authorized to perform a configuration backup or restore Verify setting for new features It is permitted to restore a configuration that was saved before applying a system update to a new version of the appliance software In some cases services and properties that were present at the time ofthe backup may have different effects and new services and properties may exist in the newer software that did not exist at the time of the backup Similar to the system update process the configuration restore process will make every effort to transfer applicable settings and apply reasonable defaults to those properties that
196. with the intended location on the motherboard and carefully insert it into its slot Slide the back ofthe riser into the motherboard rear panel stiffener Tighten the screw that secures the riser to the motherboard 136 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 7x20 Replace the rear panel PCI crossbar by sliding it down over the PCIe risers ensuring the crossbar is secured with two captive Phillips screws 9 7420 To replace the PCIe card Disengage the PCIe card slot crossbar from its locked position and rotate the crossbar into an upright position Remove the retaining screw that holds the PCIe card to the chassis Carefully remove the PCIe card from the connector and clean the slot with filtered compressed air if necessary Install the replacement PCIe card into the PCIe card slot Install the retaining screw to hold the PCIe card to the chassis Return the crossbar to its closed and locked position Chapter2 Hardware Maintenance 137 7x20 10 7120 or 7320 Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 To install the top cover Place the top cover on the chassis so that it hangs over the rear ofthe storage controller by about an inch 2 5 cm then slide the top cover forward until it seats Close the fan cover and engage the fan cover latches The cover must be comp
197. ww oracle com us products servers storage storage nas overview index html for the most recent component specification Front Panel The ZS3 2 controller drive slots and front panel components are shown in the following figure Drive slots 0 and 1 have two mirrored 900GB SAS 2 boot drives Up to four 1 6TB flash read optimized Readzilla solid state drives SSDs fill slots 2 through 5 in order Slots 6 and 7 are empty and must contain drive fillers Chapter2 Hardware Maintenance 29 253 2 Figure Legend 1 Locator LED button white 5 Service Processor OK LED green 2 Service Action Required LED amber 6 Fan CPU Memory Service Required LED 3 Power OK LED green 7 USB 2 0 ports 4 Power button Note The LEDs below the Fan CPU Memory Service Required LED are not currently used Rear Panel The ZS3 2 controller PCIe slots and rear panel components are shown in the following figure Figure Legend 1SAS 2 HBA slot 1 7 AC power supplies PS1 top PSO bottom 13 Alarm port DB 15 connector 2 4x4 SAS 2 HBA slot 2 8 System status LEDs 14 16 Cluster I O ports 3 PCIe slot 3 9 USB 2 0 ports 17 10 Gbit Ethernet ports 4 PCIe slot 4 10 SP 15 pin VGA video port 18 Chassis ground post 5 PCIe slot 5 11Serial management port 6 PCIe slot 6 12 Network management port Note The three Cluster I O ports 0 1 and GigE are reserved for cluster interconnection only 30 Orac
198. y 10 Push the DIMM into the connector until the ejector tabs lock the component in place 11 To replace the cover Chapter 2 Hardware Maintenance 81 Z53 4 Push the memory riser module into the associated CPU memory riser slot until the riser module locks in place Place the top cover on the chassis so that it is forward ofthe rear ofthe storage controller by about an inch 2 5 cm Slide the top cover toward the rear of the chassis until it seats and press down on the cover with both hands until both latches engage 82 Oracle ZFS Storage Appliance Customer Service Manual January 2014 E48492 02 ZS3 4 12 Push the release tabs on the side of each rail and slowly push the storage controller into the rack The following image shows the chassis 13 Connect the power cords to the power supplies 14 Verify that standby power is on indicated by the Power OK status indicator flashing on the front panel about two minutes after the power cords are plugged in 15 Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit The Maintenance gt Hardware screen of the BUI provides status of the replacement on the Details page for DIMMs Chapter 2 Hardware Maintenance 83 Z53 4 PCle Cards and Risers Go to the Maintenance gt Hardware screen of the B
199. y and the recommended action for the administrator if any In the CLL only the uuid diagnosed severity type and status fields are considered stable Other property values may change from release to release For hardware faults you may be able to select the affected hardware component to locate it on the Hardware screen Repairing problems Problems can be repaired by performing the steps described in the suggested action section This typically involves replacing the physical component for hardware faults or reconfiguring and restarting the affected service for software defects Repaired problems no longer appear on this screen While the system typically detects repairs automatically in some cases manual intervention may be required If a problem persists after the affected components have been repaired contact support You may be instructed to mark the problem repaired This should only be done under the direction of service personnel or as part of a documented Oracle repair procedure Related features A persistent log of all faults and defects is available under Logs as the Fault log Faults and defects are subcategories of Alerts Filter rules can be configured to cause the appliance to email administrators or perform other actions when faults are detected Chapter 3 System Maintenance 225 Logs Logs 226 Introduction Alerts This is the appliance alert log recording key events of interes
200. y a particular location or batch where the component was manufactured Part Component part number or core factory part number The orderable part number may differ depending on whether a component is for replacement or expansion and whether it s part of a larger assembly Your service provider should be able to refer you to the appropriate orderable part For components without part numbers the model number should be used instead Serial Component serial number Revision Firmware or hardware revision of the component Size Total memory or storage in bytes Type Disk type Can be one of system data log cache or spare When a spare is active it will be displayed as spare Speed Processor speed in gigahertz Cores Number of CPU cores GUID Hardware global unique identifier Disks Disks support the additional options Action Description Locate 7 Toggle the locate indicator for the disk If the LED is currently turned on this icon will be blinking Chapter 1 Introduction 19 Hardware 20 Action Description Offline o Offline the disk This option is only available for disks that are part of a configured storage pool including the system pool Offlining a disk prevents the system from reading or writing to it Faulted devices are already avoided so this option should only be required ifa disk is exhibiting performance problems that do not result in pathological failure It is no
201. y be replaced by trained Oracle service technicians PCIe Risers Each riser supports two PCIe cards that are customer replaceable There are two risers per system each attached to the rear of the motherboard Motherboard The motherboard is a FRU and includes CPU modules slots for 18 DIMMs memory control subsystems and the service processor SP subsystem The SP subsystem controls the host power and monitors host system events power and environmental The SP controller draws power from the host 3 3V standby supply rail which is available whenever the system is receiving AC input power even when the system is turned off Power Distribution Board The power distribution board is a FRU and distributes main 12V power from the power supplies to the rest of the storage controller It is directly connected to the connector break out board and to the motherboard through a bus bar and ribbon cable It also supports a top cover interlock kill switch The power supplies connect directly to the power distribution board Connector Break Out Board The connector break out board is FRU and serves as the interconnect between the power distribution board and the fan power boards storage drive backplane and I O board It also contains the top cover interlock kill switch Fan Power Boards The two fan power boards are FRUs and carry power to the system fan modules In addition they contain fan module status LEDs and transfer I2C data for

Oracle ZFS Storage Appliance Customer Service Manual

Contents

Download Pdf Manuals

Related Search

Related Contents