Content Cloud Storage with Red Hat Storage
Contents
1. Client threads Nodes gt ees 2o ead 22229 E te a DO 76 27 9 1024KB iozone file transfer size 8GB Figure 17 RandomRd Throughput Scales Linearly as Storage Nodes and or RHS Clients are added ozone record size 15 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Random Write Scalability Distributed Volume MBps vs RHS Nodesvs RHS Clients g Ww a a m non 2 me P ER py gt Ss C lien 7 16 m8 m16 824 m32 jozone record size iozonefile transfer size 8GB Figure 18 RandomWr Throughput Scales Linearly as Storage Nodes and or RHS Clients are added Test Results Replicated Volumes Throughput MBps vs Request Sz KB 4096 ewmme RandRd RandWr 2048 64 128 256 512 1024 Figure 19 RHS Replicated Volume 2 nodes 32 RHS clients Rd Wr performance saturates at reqsz gt 64KB Chart in Figure 15 illustrates the effect of request size kiloBytes variation lOzone record size on RHS Mirrored volume Throughput megabytes per second 32 lOzone client threads were used to measure throughput in this configuration Test results for this 2 node configuration along with the Distributed Replicate 4 Node configuration are presented together starting on the next page Test Results Distributed Replicated Volumes Chart in Figure 15 illustrates the effect of request size kiloBytes variation2. and replicate mirrored flavors of RHS volumes specifically 1 2 3 4 node configurations of Distribute Volumes 2 A2 node Replicate Volume 3 A 4 node Distribute Replicate Volume Before we do make sure that you have glusterd daemon running on each of rhsO1 02 03 04 If all is well you should be able to query your gluster peers and get something on the lines of what s shown below You should see Connected for status on all the peers If not check your network firewall settings and make sure that TCP ports 111 24007 24008 24009 24009 number of bricks across all volumes are open on all of rhsO1 02 03 04 If you will be using NFS also open ports 38465 38466 and 38467 gluster peer status ostname rhs82 uid 9ce1a894 71e3 4639 85e8 a3771f 727a7f Port 24887 uid e71e9136 f711 4967 93ab9 47e756ee6187 State Peer in Cluster Connected ostname rhs 4 Port 24887 uid 817c575e 7361 4b d bdf f d 98GaGaeb6a 7 State Peer in Cluster Connected Once all peers are responding we can go ahead and create our volumes For example to create a Distribute Volume named distvol4 with bricks from all 4 of our RHS server nodes do the following from any one of rhsO1 02 03 04 nodes rhs 1 rhs bricki distvol4 rhs 2 rhs bricki distvol4 rhs 3 7rhs brickivdistvol4 rhs 4 rhs bricki distvol4 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage On sim
3. Figure 22 the drop in this case is small and the advantage of using Distribute replicate combination is evident Performance Considerations This section describes the recommended system configuration settings in order to maximize performance in the Content Cloud use Case Red Hat Storage Defaults Red Hat Storage comes preconfigured with the following settings the defaults work well for the Content Cloud Large File Store use Case 1 TCP auto tuning tuned socket buffer sizes 2 Deadline I O scheduler https www kernel org doc Documentation block switching sched txt 3 Reduced Linux kernel dirty page write back frequency dirty_background_ratio from 10 percent to 1 percent Tuned Configuration Tuned is a daemon that monitors the use of system components and dynamically tunes system settings based on that monitoring information Dynamic tuning accounts for the way that various system components are used differently throughout the uptime for any given system Tuned monitors CPU network storage activity and reacts to changes in their use ths high throughput tuned profile is recommended for Sequential file access This profile helps optimize a multi stream workload at the expense of latency tuned adm profile rhs high throughput Networking Configuration 10GbE infrastructure coupled with Intel 10GbE network interface cards will yield the best performance from RHS Jumbo frames MTU 9000 and LACP 802
4. of acquisition and maintenance Red Hat Storage is a revolutionary Step forward in data management The complete elimination of centralized metadata server and the use of the Elastic Hashing Algorithm is at the heart of many of Red Hat Storage s fundamental advantages This dramatically reduces the risk of data loss data corruption and increases data availability Red Hat Storage can be deployed in the private cloud or datacenter via Red Hat Storage ISO image installed on standard server hardware based on Intel Xeon E5 2600 series processors paired with standard storage hardware resulting in a powerful turn key massively scalable and highly available storage environment Extensive testing on the reference architecture deployment discussed in this paper helps prove these benefits by demonstrating the ease of Red Hat Storage setup on standard Intel hardware and shows how Red Hat Storage performance scales linearly with the addition of storage nodes or with the number of concurrent client requests for data For more information about Red Hat Storage services and product offerings visit http www redhat com storage Glossary CIFS Common Internet File System Also known as Server Message Block SMB is a network protocol used to provide Shared access to files printers serial ports and miscellaneous communications between nodes on a network typically a set of Microsoft Windows servers and PC clients See http en wikipedia
5. org wiki Server Message Block Deduplication Data deduplication or Dedup is a specialized data compression technique for eliminating coarse grained redundant data typically used to improve storage utilization In the deduplication process duplicate data is deleted which leaves only one copy of the data to be stored along with references to the unique copy of data Deduplication reduces the required Storage capacity since only the unique data is stored From http en wikipedia org wiki Data_ deduplication FUSE File System in User Space kernel module which allows non privileged users on a Linux operating system to create a file system on the local machine HTTP The Hypertext Transfer Protocol HTTP is a networking protocol for distributed collaborative hypermedia information systems HTTP is the foundation of data communication for the World Wide Web See http en wikipedia org wiki HT TP JBOD Just a Bunch of Disks Mdadm Multiple Device Administrator is a Linux utility to manage software RAID 19 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Metadata Metadata is loosely defined as data about data Metadata is a concept that applies mainly to electronically archived or presented data and is used to describe the a definition b structure and c administration of data files with all contents in context to ease the use of the captured and archived data for further use From http
6. rhs_system_init sh is used to create bricks automagically and configure the RHS system for optimal performance In order to create a brick first identify your RAID6 volume using the Ivmdiskscan utility say dev sdX As the next step we use Logical Volume Manager LVM to create a physical volume PV Use of LVM is recommended however if you do not wish to use LVM to create logical volumes on your RAID device you may skip to the last step in this section which is formatting the device with XFS filesystem Create PV rhsOl pvcreate dataalignment 2560k dev sdX Create a volume group named RHS_vg1 mins weeweecs INAS wel clew seb Verify volume group creation with tnc Olle jowvchisjoley Make certain your RAID device is shows up in pvdisplay output rhs0Ol vgdisplay Volume group UG Name System ID Format lyumZ Metadata reas 1 Metadata Sequence No Z UG Access read write UG Status resizable MAX LY 8 Cur LU Open LU Max PU Cur PU Act PU RHS_vg1 VG Size PE Size Total PE Alloc PE 7 Size Free PE Size UG UUID 1 27 28 TiB 4 66 MiB 7152554 68679678 7 7 23 19 TiB 1872884 7 4 89 TiB p8GyxdZ XowG mdy3 Hwz4 616U FFSs 2sPD1j Create a logical volume named RHS_ v1 under the volume group RHS_vg1 and verify with lvdisplay kael dcread Il Song sm IGS divdl Rusts wel rhs01l lvdisplay Logical volume LU Path LU Name UG Name LU UUID f
7. take on a larger piece of this data pie Content clouds are emerging to provide a range of IT and consumer services for rich media consumption and distribution requiring unprecedented amounts of data to be continually stored and managed To meet this growing demand cost effectively companies have turned to Standard Intel Xeon processor based server platforms Red Hat and Intel have created such a content cloud storage reference design bringing together Intel Xeon processor based servers and Red Hat s software defined storage platform to speed organizations ability to rapidly implement content clouds The goals for this reference architecture are the ability to scale out in a modular incremental manner based on evolving demands being optimized for storage and management of unstructured data and being architected for cost effectiveness based on Linux and Intel technology without the need for custom hardware Red Hat Storage Product Brief Red Hat Storage RHS is a software only open source scale out solution designed to meet unstructured semi structured and big data storage requirements At the heart of Red Hat Storage is GlusterFS an open source massively scalable distributed file system that allows organizations to combine large numbers of storage and compute resources into a high performance virtualized and centrally man aged storage pool With Red Hat Enterprise Storage enterprises can turn Intel Xeon processor based serv
8. topology hardware and software deployed installation and configuration steps and tests for real world use cases that should significantly reduce the learning curve for building and operating your first cloud infrastructure It should be noted that the creation and operation of a storage cloud requires significant integration and customization to be based on existing IT infrastructure and business requirements As a result it is not expected that this paper can be used as is For example adapting to an existing network and identifying management requirements are out of scope for this paper Therefore it is expected that the user of this paper will make significant adjustments to the design presented in order to meet their specific requirements This paper also assumes that the reader has basic knowledge of cloud storage infrastructure components and services Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Table of Contents SPAAN EEE ene ee ae ee oe ee ee he ern a ee ee ee en eee re ee E ee ee ee ere 3 MOU ON airean Tna ns apaenepesecsewadsecs san eoneentar ces ceunnanoneeyone TEE AE OE ENEA ARE EAEE ATE 3 Red Hat Storage Product Brief 255 anne neeoins neers ii iue AAE RAA 3 Red Hat Storage Technical Overview sssssssrssssererornrnrrrnrr rnnr rnnr ews bea ee PENE EEEE a eee EREE 4 Red Hat Storage Possible Configurations lt asswapepeniciweavaecawineanperendesaes sar eiesee ae oiedey asker ehanseipe
9. 31872 hs i mirrorvolZ on mnt fuse mirrorvol2 type fuse glusterfs rw default_permis fons allow_other msax_read 131872 hs i mirrorvol on mnt fuse mirrorvol4 type fuse glusterfs rw default_permis sions allow_other sax_read 13187Z The clients are now ready to use the RHS cluster Access to the cluster is provided via the glusterfs mount points mnt fuse shown above Performance and Scalability Testing with lOzone We use lOzone to test the performance of our reference RHS cluster To recap we have 4 RHS server nodes each is hosting a RAID6 based RHS brick and is running glusterd We have created 5 RHS volumes 3 Distribute distvol2 distvol3 distvol4 1 Replicate mirrorvol2 and 1 Distributed Replicate mirrorvol4 We use lOzone on up to 8 RHS native clients for IO performance scalability testing lOzone Configuration lOzone is run in Cluster testing mode to launch 4 lOzone tester threads on each client With 8 clients that gives us the ability to test with 8 16 24 or 32 threads performing IO to the RHS cluster concurrently The lOzone command line used is rhs clientl iozone m S IOZONE C CONbiIc FILENAME 1i S IOZONE E TEST n C u n TS S IOZONE EIES E 2 IOZONE T RECORDSZ zA Cage aoe S TEST THREADS R Di result exis where m specifies cluster testing mode IOZONE_CONFIG_ FILENAME is the lOzone config file for the cluster test format that lists the clien
10. 3ad bonded interfaces will help with a jump in the cached read throughput With jumbo frames with native GlusterFS client access for IO sizes gt 64KB one should expect to use most of 10GbE pipe Storage Array RAID6 Configuration Larger hardware RAID stripe element size utilizes the hardware RAID cache effectively and can provide more parallelism at RAID level for small files because fewer disks would be required to participate in the read operations For RAID 6 the recommended Stripe size is 256 KB LVM Configuration If using LVM on top of RAID6 for creating RHS bricks the dataalignment option to pvcreate helps to make sure that IO is aligned at the RAID6 stripe boundary The value is calculated by multiplying the RAID6 stripe size by the number of data disks in the RAID volume As noted in the cookbook section we used 12 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage disk RAID6 configuration 10 data 2 parity disks with 256KB Stripe size This translates to dataalignment size of 2560KB pvcreate dataalignment 2560k lt RAIDDEVICE gt XFS Filesystem Configuration XFS allows the selection of logical block size for directories that is greater than the logical block size of the file system Increasing the logical block size for the directories from default 4K to 8K decreases the directory IO this improves the performance of directory operations For example a MRES XES L Lze
11. 512 n Si wesley d su 256k sw 10 dev lt RHS VG gt lt RHS_ LV gt XFS Mount Options Most applications do not require an update to the access time on files or directories It is recommended that RHS bricks are mounted with the noatime and nodiratime option Also inode64 option allows inodes to be placed near to the data which minimizes disk seeks Entry in fstab jo ew EA G2 e PE 97 tls lor lie E inode64 noatime nodiratime 1 2 RHS GlusterFS Client Access Method Native Client is a FUSE based client running in user space Native Client is the recommended method for accessing Red Hat Storage volumes when high concurrency and high write performance Is required For additional Linux performance tuning guidelines see Red Hat Linux Performance Tuning Guide Other Things to Consider Power Optimization Power consumption for large amounts of storage i e Petabyte 1015 or even Zettabytes 1021 can become a significant part of the total cost of ownership for a storage cloud The scale out Storage cloud test could be augmented by profiling the power consumption of the racks when idle power aware workload virtual machine scheduling or being tested by the usage models The power utilization could be optimized through technology such as Intel Intelligent Power Node Manager Conclusion By delivering increased scalability flexibility affordability performance and ease of use in concert with reduced cost
12. 6 parameters Set caching policy to cached rhs0l CmdTool264 LDSetProp Cached TALL a0 Set disk cache policy to enabled rhs01 CmdTool264 LDSetProp EnDskCache LALL a0 Enable write policy to WriteBack chasOl CmdTool264 LDSeerProp WB LALL aALL The WriteBack policy enables asynchronous writes and thus improves performance For this mode to work reliably battery backup for Write back cache Is essential For other RAID controller configuration commands please refer to the LSI Intel MegaRAID adapter user manual To watch progress of RAID6 virtual drive creation rhs0O1 CmdTool264 AdpAlILog aALL To confirm RAID6 virtual drive creation rasol CmdloolZ64 EDITO TALL AALL Here is the LDInfo output from our setup dapter 8 Virtual Drive Information Virtual Drive 8 Target Id 8 Name RAID Level gt Primary 6 Secondary 8 RAID Level Qualifier 3 Size 27 284 TB Parity Size 5 456 TB State Optimal 256 KB 12 D WriteBack Readfdaptive Cached Write Cache OK if Bad BBU WriteBack ReadAdaptive Cached Write Cache OK if Bad BBU Read Write Read Write Enabled None PI type No PI Is UD Cached No Red Hat Storage Node Brick Creation A brick is the unit of storage in RHS One or more bricks make an RHS volume The brick creation steps provided in this section are for the reader s reference only and typically a script
13. DING OPTS miimon 188 mode 4 lacp_rate 8 xmit_hash_policy 2 MTU 9888 root rhs 1 18 cat etc sysconf igznetwork scriptszifcfg eth8 DEVICE cth NM_CONTROLLED no MASTER bond SLAVE yes MTU 9886 froot rhs61 IN _ Note the parameter BONDE NG Orie oh MeCe Ie site x1 Tienes hive policy 2 This setting tells the Linux bonding driver to configure the bondO interface in 802 3ad LACP mode which is mode 4 in Linux terms The parameter xmit_hash_policy 2 configures the bonding driver to use a hashing function on the network Layer 2 and Layer 3 packet header fields in order to achieve better load balancing across the slave 10GbE interfaces that are part of bondO Also recommended is the setting MTU 9000 This is to enable Jumbo Frames Please note that if jumbo frames are enabled on the server nodes the same MTU should be configured on all nodes client and any other test nodes for best results Once the interfaces ethO eth1 and bondO are configured restart RHS network service Verify bonding settings with the commands shown here Lroot rhs61 18 cat sys class net bond amp bonding mode 662 3ad 4 froot rhs 1 1 cat sys class net bond bonding xmit_hash_policy layerZ 3 Z Croot rhs 1 J cat sys class net bond6 bonding slaves eth8 ethi root rhs6 1 18 cat sys class net bond amp bonding lacp_rate slow 6 Red Hat Storage Node Storage Configurati
14. ION WITH INTEL PRODUCTS NO LICENSE EXPRESS OR IMPLIED BY ESTOPPEL OR OTHERWISE TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT EXCEPT AS PROVIDED IN INTEL S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE MERCHANTABILITY OR INFRINGEMENT OF ANY PATENT COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT UNLESS OTHERWISE AGREED IN WRITING BY INTEL THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR Intel may make changes to specifications and product descriptions at any time without notice Designers must not rely on the absence or characteristics of any features or instructions marked reserved or undefined Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them The information here is Subject to change without notice Do not finalize a design with this information The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications Current characterized errata are available on request Contact your
15. Intel Cloud Builders Guide Intel Xeon Processor based Servers Content Cloud Storage with Red Hat Storage intel Intel Cloud Builders Guide to Cloud Design and Deployment on Intel Platforms Content Cloud Storage with Red Hat Storage nt P inside ntel e Intel Xeon Processor E5 2600 Series RED HAT STORAGE Tushar Gohad Senior Staff Engineer Storage Division Intel Corporation Veda Shankar Principal Technical Marketing Manager Red Hat Storage Red Hat Inc June 2013 Audience and Purpose For companies looking to build their own cloud computing infrastructure including both enterprise IT organizations and cloud service providers or cloud hosting providers the decision to use a cloud for the delivery of IT services is best done by starting with the knowledge and experience gained from previous work This white paper gathers into one place the essentials of a scale out storage cloud reference architecture coupled with an example installation of GlusterFS the foundational component of Red Hat Storage RHS optimized for the Content Cloud Storage Large File Object Store use case This workload is characterized by a large number of concurrent users accessing Storage with a write once read often access pattern The reference architecture based on Intel Xeon processor based servers creates a multi node capacity optimized cloud Storage deployment The white paper contains details on the cloud
16. KWO83f xNuJd f AUW pUNM rcPF 98v 0A069L LV Write Access read write LU Creation host time rhs 1 2813 85 87 89 37 59 8788 LU Status available dev RHS_vg1 RHS_lv1 RHS_ ivi RHS_vgi open LU Size Current LE Segments 1 23 19 TiB 6879678 Allocation inherit Read ahead sectors auto currently set to 256 Block device 253 3 11 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Next format the logical volume RHS_lv1 with XFS filesystem We ll use this XFS formatted logical volume as one of the bricks Create a mount point for the brick and create an etc fstab Mount the newly created filesystem The XFS volume mounted under rhsO1 rhs brick1 is our first brick that we can use to create an RHS volume in the next section Before we go there we need to repeat this brick creation process on all RHS server nodes rhsO2 rhsO3 rhsO4 On each of rhsO1 02 03 04 start glusterd restart if already running Take this opportunity to also set up tuned on each of rhs01 To make this setting persist across reboots you may want to use etc tuned conf Please see tuned conf 5 and tuned adm 1 man pages for more information Red Hat Storage Node Volume Creation Once we have created and mounted bricks on all RHS server nodes we ll instantiate a few RHS volumes that are useful for recreating the test setup we describe here We create distribute
17. Network Intel X520 T2 Converged 192 168 92 106 10GbE Ethernet 192 168 92 108 Load Clients 1 8 GlusterFS Native Access Figure 11 RHS Reference Architecture Content Cloud Storage Array Reference Configuration We used the Newisys Sanmina NDS 4600 4U 60 SAS hard drive JBOD as the external storage array to host the twelve 3TB SAS drives per RHS server node The JBOD drive array was partitioned such that only 12 SAS drives are presented to each RHS server node this is known as Zoning Configuration 3 in the NDS 4600 JBOD terminology Please refer to JBOD manual for further details on how to configure this mode RHS Client Test Configuration The test setup included up to 8 RHS load generator clients shown in Figure 11 Each client was configured for native GlusterFS access Hardware configuration for each client was Intel Xeon Processor E5 2680 with 8 CPU cores running at 2 GHz 64GB DDR3 1600 ECC protected memory and an Intel X520 T2 10GbE Ethernet adapter Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Test Network Configuration 10GbE LACP Jumbo Frames The RHS cluster and the test clients were networked using an Arista Networks 7O50T fully non blocking manageable 10GbE Ethernet switch The switch ports were configured so that IP packets destined to test network 192 168 92 0 24 are addressed to the same VLAN Network switch ports were configured to Support jumb
18. age 2 x Native Client RHN channel Install Native RHS Client rpms Load FUSE module Verify that FUSE module is loaded Key based SSH Setup between Server Client Nodes Set up key based authentication between all the RHS server and client nodes Do the following on each node Generate SSH key pair Copy the keys to other hosts Repeat on RHS nodes rhsO 2 4 and clients rhs client 1 8 1 UW Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Mount RHS Volumes on Client Nodes Next mount all RHS volumes we created in section Red Hat Storage Node Volume Creation using glusterfs filesystem type native RHS client access For example to mount distvol4 on rhs client1 rhs clienel Mkdir p mne fuse cisteyol4 raS clientii mounte t glusteris i Enel drusrvol4 m mnt ruse dist yol Repeat on all clients rhs client1 rhs client8 for all volumes distvol2 distvol3 distvol4 mirrorvol2 mirrorvol4 Verify native RHS mounts on each client rhs clientl mount grep gluster eC n t sit Croot rhs clienti rhs_tools mount grep gluster hs i distvol2 on mnt fuse distvold type fuse glusterfs Crw default_permission ollow_other max_read 131872 hs i distvol3 on mnt fuse distvold type fuse glusterfs Cru default_permission ollow_other max_read 131872 hs i vdistvol4 on mnt fuse distvold type Fuse glusterfs rw default_permission eallow_other max_read 1
19. designed to perform across a wide variety of workloads and was designed to enable customers to economically achieve very good performance levels under less than ideal conditions It provides enterprises the ability to easily adjust configurations to achieve the optimal balance between performance cost manageability and availability for their particular needs Red Hat Storage uses advanced performance enhancing techniques to provide a predictable performance Some of the techniques include write behind asynchronous I O intelligent I O scheduling and aggregated operations Unified File and Object Access RHS supports FUSE based native client access to GlusterFS volumes Native client is the recommended method for accessing GlusterFS volumes when high concurrency and high write performance is required RHS also supports CIFS and NFS protocols for providing standards based access methods to its scaled out volumes Via the Unified File and Object UFO interface RHS supports object access using OpenStack Swift API It is also the first product that enables unified access to objects as files through Standard NFS and CIFS mounts as well as access to files created on GlusterFS volumes as objects with the OpenStack Swift protocol HDFS Compatibility HDFS support in Red Hat Storage provides stronger protection against failure of data and name nodes in a Hadoop deployment Large and long running data processing benefits from the combina
20. e RHS Server Node Reference Configuration Figure 11 RHS Reference Architecture illustrates the test bed setup that we used for experiments performance measurements As shown we deployed RHS in a 4x Storage Node cluster This RHS Storage Server portion of the test bed is detailed in Figure 10 RHS Server Node Reference Configuration see nodes rhsO1 rhsO2 rhsO3 and rhs04 A high density 2U 4node Intel H2300 F system provided us with an optimal form factor for RHS server setup Each storage node was equipped with Intel Xeon Processor E5 2680 with 8 CPU cores running at 2 7GHz 64GB DDR3 1600 ECC protected memory dual ported Intel X520 T2 1OGbE Ethernet adapter and an Intel RS25SBO08 SAS 6Gbps RAID controller hardware RAID support with battery backup Each RHS server node was connected to an external storage array with twelve 3TB Seagate SAS 10k hard drives configured in RAID6 mode This provides a total of 48 3TB 144TB of storage in a 4 node RHS system This combined with the 2U form factor makes this configuration optimal from a storage capacity per rack perspective Xeon E5 2600 Compute Density Upto 8 Intel Xeon Processor E5 2600 per 2U system via 4 hot swappable half width nodes Intel H2300JF Supermicro 2UTwin2 CPU 2x Intel Xeon Processor E5 2680 8cores 2 7GHz 20MB 130W Memory 64GB DDR3 1600 ECC Available as E P Sa aay J Dell PowerEdge C6220 i Motherb
21. e eee eaee neue 5 Red Mat StOrade USC LISOS o2cccaveveassancdaetetenceeesventaniereteatease qo veaxteseneersadenpnnsceeesscesd 4aneaeuaewRrnaense lt 5 Content Cloud Large File and ODJEcCtStOrE 55 5 45 yaaa ney enee oaaoes ieee nae ead ee rks aes este Paes yeaeuseanesieaes 6 Enterprise Drop Box or Cloud Storage for Service Providers ccc cece cece cece eee eee eeeeeeereevrusseeeeteeeusegeeenness 6 Red Hat Storage Reference Architecture for Content Cloud USAGES ccc cece tener teen eee nee eee teen enenneees 7 Red Hat Storage Server Node Reference CONFIGUFATION ccc ccc eee cee e eee e eee eee eee bette e ene tte eebateennaes 7 Storage Array Reference Configuration o454rcsae enw iaeeases acid Rienihe unas aean ee UN rnrn rs axG see Teen eee ers 8 ked Aat storage Sel UIMACOOKDOOK sssrini Arrn te bee hoee ie gaee assent test eubecereeseeeecsedgetaasegaeantessaseneet ae 9 Performance and Scalability Testing with OZONG oi 45 00 cane atone wdeoaveen pena i0wsG ees bas Han G nee rend ere ak bey eae oesee eas 9 OZONE CONO O 5 ases one ee e eet ane ee ae neeroee ee te eae eas ate AE sie shea apa eea AERA SAER 14 TOSTE TOC OMA OS dsnecesccanneaceranese sees rnett ea teadseuenececewwaa sence eaesenuaoeanceanacnewaeesseceneseesaecetenerss 14 Test RESUS DISTIDUTEG VOIUMES ses ssnccautacdnsersne sinSns cAvctan Sin aged AATE ATRA RAR EANA DAENT EAD 14 Test Results Replicated VOWS 214540020350 907iararsdeeiaderrncatadsseqneeeeededdaiestssetdead
22. e re iozone record size 1024KB iozone file transfer size 8GB Figure 22 Seqwr Throughput Scales upward Linearly as Storage Nodes are added however exhibits decline as more RHS Clients are added this decline may be attributed to added system pressure and back end network traffic due to replication Distributed Replicated Mode Random Read re Client threads Nodes gt Oo 8 201 538 B ET pts ES soss Esso pk a 1 1 a Eee lozone record size 1024KB iozone file transfer size 8GB Figure 23 SeqRd Throughput Scales Linearly as Storage Nodes and or RHS Clients are added 17 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Random Write Scalability Distributed Replicated Volume MBps vs RHS Nodesvs RHS Clients a w _ 2 a lt o 2 x lt Ch ene 16 8 FR 32 ms 216 n24 m32 Distributed Mode Random Write Performance Scalability Po Cluster Size nodes vs Throughput MBps Client threads Nodes gt 2 4 Pts TS 82925 ee e cE E A iozone record size iozone file transfer size 8GB Figure 24 Seqwr Throughput Scales upward Linearly as Storage Nodes are added however exhibits decline as more RHS Clients are added this decline may be attributed to added system pressure and back end network traffic due to replication When compared to the drop in performance in the replicate only configuration
23. en wikipedia org wiki Metadata Namespace Namespace is an abstract container or environment created to hold a logical grouping of unique identifiers or symbols NAS Network Attached Storage is a storage server or appliance that uses file based protocols such as NFS Network File Server or CIFs to enable clients typically servers and PCs to access files over a TCP IP network See http en wikipedia org wiki Network attached_storage NFS Network File System protocol designed to let client access files on the server over the network connection RAID Redundant Array of Independent Disks Replication Data replication is the process of sharing data so as to improve reliability between redundant storage devices The replication is transparent to an application or end user In a failure scenario failover of replicas is hidden as much as possible REST Representational State Transfer REST is an architecture for client server communication over a TCP IP network e g Internet Clients initiate requests to Servers Servers process requests and return appropriate responses At any particular time a client can either be in transition between application States or at rest A client in a rest state is able to interact with its user but creates no load and consumes no per client storage on the set of servers or on the network The client begins to send requests when it is ready to make the transition to a new State The Hypertext Tran
24. eras The total size of the data store Directly proportional to the number of users this service is offered to They typically start at hundreds of terabytes as there is a need to pre provision for potentially millions of subscribers customers Such data stores can grow up to several petabytes over time where the data can Span multiple data centers Data access pattern Hundreds of concurrent users reading and writing at the same time The storage solution must be able to deliver sustained high throughput The RHS advantage Scale out architecture provides massive and linear scalability Capacity and performance can scale independently and linearly with no bottlenecks Object storage support via OpenStack SWIFT API Support for multiple Dropbox software licenses or seats that can be co resident on the storage servers Storage and retrieval of files and objects interchangeably Consistent sustained predictable throughput performance Sizing Recommend dual socket Intel Xeon 5 2600 series processors with 64GB of DRAM for storage node system memory Red Hat Storage Reference Architecture for Content Cloud Usages One of the goals of this paper is to define a reference configuration for a Red Hat Storage cluster that is optimized on Intel architecture for the Content Cloud large file object store use case Towards this goal we experimented with RHS in an elaborate test setup that led to a reference architectur
25. ers and storage resources either on premise or in the public cloud into a scale on demand virtualized and centrally managed storage pool The global namespace capability aggregates disk CPU I O and memory into a single pool of resources with flexible back end disk options supporting direct attached JBOD or SAN storage Storage server nodes can be added or removed without disruption to service enabling Storage to grow or shrink on the fly in the most dynamic environments Salient features of Red Hat Storage include POSIX like Distributed File System Red Hat Storage is based on GlusterFS which is a powerful distributed filesystem written in user space which uses FUSE to hook itself with Linux VFS layer on top of standard Linux filesystems such as XFS GlusterFS is POSIX compliant SE Files and folders Documents images audio video large archives Big data Log files RFID data Virtual machine images Figure 1 What can be stored in RHS Global Namespace The global namespace capability aggregates disk CPU I O and memory from multiple servers into a single pool of resources with flexible back end disk options Supporting direct attached JBOD or SAN storage Storage server nodes can be added or removed without disruption to service enabling storage to grow or shrink on the fly in the most dynamic environments Scale out performance capacity and availability RED HAT STORAGE FOR ON PREMISE Scale
26. ghts Reserved t Create RAID Volume 3 Reset Disks to Non RAID Z Delete RAID Volume Exit ee eee Volumes ne defined Physical Devices ID Device Model ST966020555 Serial IEN LO ODQOOMJ3J06V5H IEQGOOGORJIJLJAKT 558 9GR Size TypesStatus Vol 58 96B al p Compright cc 2003 12 Intel cabasent ties ALI Rights Reserved Name Volume RAID Level RAIDICMirror Serial Size Status Figure 13 RAID1 Volume Creation for RHS 2 0 install In the next step we install Red Hat Storage using the ISO image RHS 2 x Update y Installation DVD You can download the ISO using your Red Hat Network account a Red Hat Storage Server 2 0 RHEL 6 2 z for x86_64 es Subscribed Systems Target Systems Downloads ISO Image Downloads NOTE By downloading this software you agree to the terms and conditi of the applicable License Agreement available at http www redhat com licenses 2 Not sure how to download and use these images Check out our ISO Download Help amp Latest Release e find the mane et of ISO in rikn j 2 0 RHEL 6 2 Zi x86_64 you d like to install y y on of Red Hat Storage Serv pine ed a subset of these dis informatio z for x86_64 Depending on the variant of Red Hat Storage Red Hat Storage 2 0 RHEL 6 2 z for x86_64 1 605 MB MDS 634d1e80f9ac29 679380fa67EZ2 1bb SHA 256 93 RHS 2 0 Update 4 Installation DVD amp Intel Cloud Bu
27. he filename changes a pointer file is written to the server that the new hash code would point to telling the distribute translator on which server the file is actually stored MOUNT POINT DISTRIBUTED VOLUME server2 exp1 _ FILE 1 FILE 2 FILE 3 Figure 4 GlusterFS Distribute File level RAIDO Advantages A The more servers you add the better this scales in terms of random file access As long as clients aren t all retrieving the same file their access should be spread pretty evenly across all the servers B Increasing volume can be done by adding a new Server c Adding servers can be done on the fly Disadvantages A If you lose a single server you lose access to all the files that are hosted on that server This is why distribute is typically graphed to the replicate translator B If your file is larger than the sub volume writing your file will fail C If the file is not where the hash code calculates to an extra lookup operation must be performed adding slightly to latency Replicated Volumes Replicate Replicate is a translator that takes a list of bricks and replicates files across them This is analogous to a file level RAID 1 volume FILE 1 FILE 2 Figure 5 GlusterFS Replicate File level RAID1 Distributed Replicated Volumes Distribute Replicate as the name suggests is a translator that takes a list of bricks and replicates files across them providing file distributi
28. ic unit of storage represented by an export directory on a server in a trusted storage pool A Brick is expressed by combining a server with an export directory in the following format SERVER EXPORT For example lt hostname gt rhs brickl1 Red Hat Storage Client A node that mounts a GlusterFS volume this may also be a server Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Red Hat Storage Server A node virtual or bare metal that hosts the real filesystem XFS for example where data will be Stored glusterd the GlusterFS management daemon runs on GlusterFS servers GlusterFS Translator A translator connects to one or more sub volumes does something with them and offers a sub volume connection An example of a GlusterFS translator is replicate which applies replication to bricks GlusterFS Sub volume A brick after being processed by at least one GlusterFS translator GlusterFS Volume The final GlusterFS share after it passes through all the translators configured The other view of a Volume is a logical collection of GlusterFS bricks Red Hat Storage Possible Configurations Distributed Volumes Distribute Distribute is a translator that takes a list of bricks and spreads files evenly across them effectively making one single larger storage volume from a series of smaller ones The server that the files are written to is calculated by hashing the filename If t
29. ice Providers Unified namespace Figure 9 Enterprise drop box Cloud storage for SPs SPs provide a share to end users to store content accessible from a variety of devices Service Providers SPs including telcos cable companies and other content providers are increasingly faced with competitive needs to provide differentiated services to their end customers As a consequence service providers offer their end customers Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage access to free and or paid storage for storing photos videos and other media files that they access via the web using a smart phone tablet or personal computer Building a reliable back end Storage infrastructure with traditional monolithic storage gear does not meet the agility cost point and versatility demanded by such storage deployments A similar use case is seen in the context of an enterprise where employees need to reliably and securely store and retrieve files from a variety of devices Industries include Telecom and media verticals The Enterprise Dropbox requirement applies to all verticals The workload Can be characterized as initially comprising of only writes as end users upload media files Over time expect 50 50 reads and writes as end users tend to retrieve and play those media files File sizes A few megabytes pictures taken using smart phones to dozens of megabytes HD videos taken using high end digital SLR cam
30. ilar lines we created the following volumes for our test setup 1 distvol2 only with rhsO1 and rhsO2 2 node Distribute 2 distvol3 with rhsO1 rhsO2 and rhsO3 3 node Distribute Next we create a Replicate or a mirrored volume with 2 nodes participating Verify volume creation with root rhs 1 1 gluster volume info mirrorvolZ Bricki rhs 1 rhs bricki mirrorvol2 Brick2 rhs 2 rhs bricki mirrorvol2 We create another Replicate volume with all 4 nodes and instead of setting the number replicas to 4 we set the number to 2 so RHS divides the node set into two one subset being the mirror of the other with files being distributed across nodes within a subset essentially creating a combo Distributed Replicate volume named say mirrorvol4 rhs 1 rhs bricki mirrorvol4 rhs8 2 rhs bricki mirrorvol4 rhs 3 7rhs bricki mirrorvol4 gt rhs 4 rhs bricki mirrorvol4 Make sure all the volumes are started with gluster volume info on each volume If all looks ok we are ready to mount the volumes on RHS client RHS Native Client Configuration Load Generators In test setup described in this paper 8 RHS clients were used rhs client1 through rhs client8 in Figure 11 Red Hat Enterprise Linux Server 6 4 was installed on all test clients All clients were FUSE based clients running in user space Here are detailed setup instructions for the clients Subscribe to the Red Hat Stor
31. ilders Guide Content Cloud Storage with Red Hat Storage Connect RHS nodes to a network that can provide the node Internet connectivity Boot the node s off of RHS install DVD and install Red Hat Storage with defaults recommended Once RHS install is complete use rhn register utility to register the RHS node on Red Hat Network RHN rhs0O1 rhn register Subscribe to the RHN channels RHS 2 x and RHEL Server SFS 6 x z in order to receive updates rhs0Ol rhn channel add clammcl rhicil x66 TT Ol server O rna rhs0l rhn channel add Cenan l rael e o ene Sa Update the RHS node using rhs0l yum update thsO1 is used in all examples in the remainder of this document Perform the same actions on rhsO2 rhsO3 and rhs04 Red Hat Storage Node Network Configuration Once RHS is installed the next step is to connect the node s to the 10GbE network we configured in section Test Network Configuration 10GbE LACP Jumbo Frames It is important that the node s are installed with 10Gb Ethernet adapter or adapters providing at least two 10GbE ports for connectivity Configure the two 10GDE interfaces say ethO and eth1 in a mode 4 802 3ad LACP bonding configuration in order to match the 10GbE switch port configuration Here is an example configuration from rhs01 Croot rhs81 18 cat etc sysconf ig network scripts ifcfg bondB DEVICE bond ONBOOT yes NM_CONTROLLED no BON
32. lOzone record size on throughput megaBytes per second of RHS Distributed Mirrored volume constructed with 4 RHS nodes 32 Ozone client threads were used to measure throughput in this configuration Throughput MBps vs Request Sz KB 2048 4096 ewes RandRd qx lt eRandWr 256 512 1024 SeqRd SeqWr Figure 20 RHS Distributed Replicated Volume 4 nodes 32 RHS clients SeqRd Wr performance saturates at reqsz gt 64KB RandRd RandWr scale We used 1024KB as the representative record size request size for presenting the data gathered during this experiment Throughput MBps iozone record size iozone file transfer size 8GB Figure 21 SeqRd Throughput Scales Linearly as Storage Nodes and or a a gt a Da pe O i 3 600 00 Sequential Read Scalability Distributed Replicated Volume MBps vs RHS Nodesvs RHS Clients RHS Clients are added Random Read Scalability Distributed Replicated Volume MBps vs RHS Nodesvs RHS Clients Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Sequential Write Scalability Distributed Replicated Volume MBps vs RHS Nodesvs RHS Clients 2 3 Throughput MBps 8 X 4 64 m8 816 n24 32 Replicated Mode Sequential Write Performance Scalability RHSCusterSize nodes vs Throughput MBps Client threads Nodes gt er a o Sie 393 98 7 RS i
33. local Intel sales office or your distributor to obtain the latest specifications and before placing your product order Copies of documents which have an order number and are referenced in this document or other Intel literature may be obtained by calling 1 800 548 4725 or by visiting Intel s Web site at www intel com Copyright 2013 Intel Corporation All rights reserved Intel the Intel logo Xeon Xeon inside and Intel Intelligent Power Node Manager are trademarks of Intel Corporation in the U S and other countries Other names and brands may be claimed as the property of others intel
34. nstructured data piles up at an exponential rate enterprises continue to react by adding more file servers or expensive monolithic NAS devices which over time becomes hard to manage amp control and creates a file server sprawl This in many cases results in underutilization of pricey NAS devices or lower performance because of over utilization of those file servers and NAS devices This is common in industries like oil gas exploration Telco data app logs billing healthcare x ray patient records finance risk analysis compliance and security data scientific gene sequencing simulations and miscellaneous applications such as rendering geographical information systems and computer aided engineering Workload Several concurrent users Read mostly Write sparingly Average file size Small to large a few kilobytes to several megabytes Total size of the data store Start small a few terabytes but will often grow to several hundred terabytes and achieve steady state The RHS advantage Elastic scalability Eliminate storage silos centralize amp simplify management with global namespace technology Leverage volume economics amp maximize Multi protocol client support for file sharing Sizing Recommend dual socket Intel Xeon E5 2600 series processors with 64GB of DRAM for storage node system memory Use 3TB SAS 10k rpm or 15k rpm drives for storage Enterprise Drop Box or Cloud Storage for Serv
35. o frames MTU 9216 In addition each of the two network switch ports connected to the RHS server nodes rhsO1 rhsO2 rhsO3 and rhsO4 were bonded in the dynamic link aggregation configuration with LACP link aggregation control protocol as described in IEEE 802 3ad standard A tutorial on Arista switch configuration for this mode is illustrated at https eos aristanetworks com 2011 05 link aggregation primer One of the important things to note here is that the aggregated link also called channel group should be assigned the same VLAN ID as one assigned to the test subnet 192 168 92 0 24 in the test setup described here Red Hat Storage Setup Cookbook This section describes the steps involved in hardware configuration through deploying Red Hat Storage software on Storage server nodes This cookbook style tutorial should help the reader recreate the reference architecture test setup shown in Figure 11 RHS Reference Architecture Content Cloud Red Hat Storage Node BIOS Configuration On each RHS server node make sure you have Intel Turbo Boost and Enhanced Intel SpeedStep Technology enabled Intel SpeedStep will make sure your RHS servers do not run at maximum CPU frequency and thus run hot all the time Copyright O 2010 2012 Aserican Neqatrends Aiptia Setup Utility vanced Power amp Performance Allows the user to set an Performance Optimization is strongly toward performance even at the e
36. oard S2600JFQ Network Intel X520 T2 Converged Ethernet 2x 10GbE copper ports bonded on the switch with 802 3ad OS Disk 2x 600GB 2 5 SAS 10k 64MB Seagate ST960020588 RAIDO Storage Controller Intel RS25SB008 SAS RAID Controller Card PCle x16 battery backed Storage Array via 2x external JBODs 4U 60 NDS 4600 RHS Storage Node Reference Configuration Intel H2300JF System 4 dual CPU servers in a 2U form factor Figure 10 Reference Hardware Configuration for RHS Server Nodes rhs01 rhsO2 rhsO3 rhsO4 in Figure 11 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Gluster Node rhs01 192 168 92 11 12x 3TB SAS 10k 6Gbps JBOD2 4U NDS 4600 2 x12 SAS drives O O7 12x 3TB SAS 10k 6Gbps RAID6 Red Hat Storage Cluster EE Gluster Node rhs02 10Gigabit Ethernet g 2U 4 node Server H2300JF Gluster Node rhs03 192 168 92 13 RAID6 gt 12x 3TB SAS 10k 6Gbps JBOD2 4U NDS 4600 2 x12 SAS drives 12x 3TB SAS 10k 6Gbps 192 168 92 va RAID6 Gluster Node rhs04 5A5 Connections Hho Arista Networks 7050T 10GbE Switch 192 168 92 102 192 168 92 103 CPU 2x Intel Xeon Processor l us 3 3 E5 2680 4 Motherboard ay 52600JFQ 192 168 92 101 192 168 92 104 192 168 92 107 168 92 105 ia i i O 3 h aaa ir a Lita aia 6 Ian a a p TA Csi Memory 64GB DDR3 1600 ECC
37. on RAID6 As described in section RHS Server Node Reference Configuration an external JBOD NDS 4600 provides the Storage backend to the RHS server nodes The nodes are connected via an Intel RS25SBO08 SAS 6Gbps battery backed RAID controller to the 12 disk JBOD disk array The recommended RAID configuration for the disk array is hardware assisted RAID6 with battery backed write back cache Here are instructions for recreating the setup Use the CmdTool264 utility called megacli in case of LSI branded RAID adapters to configure RAID6 on each of the RHS server nodes example configuration for one RHS node is shown here Determine JBOD enclosure ID and the slot numbers that have disks installed rhs01 CmdTool264 LdPdInfo a0 grep Enclosure Device ID rhs01 CmdTool264 LdPdInfo a0 G2ep gt blot Number Use the JBOD enclosure ID and the appropriate disk slot numbers reported by the previous set of commands to create a RAID6 volume rhs01 CmdTool264 CfgLdAdd ro 024 Ora oee TOs 10328 10 29 10 50 10 40 10341 TO Eo IOA e a SitmosZs 270 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage r6 specifies RAID6 creation 10 is the enclosure ID and 4 5 6 41 42 are the 12 disk slot numbers in the JBOD that we want to be RAID6 members The strpsz parameter specifies the RAID stripe size The recommended RAID6 strip size is 256KB Configure RAID
38. on within the replicated volume at the same time This is analogous to a file level RAID 1 0 volume FILE 1 Figure 6 GlusterFS Distributed Replicate File level RAID 1 0 FILE 2 Red Hat Storage Use Cases Red Hat Storage has broad use cases in scale out cloud environment Standardized NAS and object storage server deployments both on premise and in the Amazon Web Services AWS public cloud are possible Common solutions based on RHS include Content Delivery Network CDN Backup Disaster Recovery DR High Performance Computing HPC and Infrastructure as a Service laaS Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Live VM image store Cost effective and reliable alternative for VM storage ade e se pasn uyo SBAIYDIY JULE N i F a 3 Repicated Red Hat Storage voume in EC AWS F Red Hat ace y Bej ozur dn pinq yey eepo sonnuenb Bre ajea 88e y suogeoyddy 34035 pafqo pue jy IZel i i datacenters to T synchronized Disaster Recovery Medium to large enterprises with DR requirements and multiple Figure 7 RHS Use Cases Summary Content Cloud Large File and Object Store Figure 8 Content cloud Massive simultaneous consumption of multimedia content by thousands of users In a number of enterprises applications generate a vast amount of data either in form files and or objects that need to be stored for analysis and synthesis at a later time The u
39. ributed volume Throughput megaBytes per second 32 lOzone client threads were used to measure throughput in this configuration We used 1024KB as the representative record size request size for presenting the data gathered during this experiment Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Sequential Read Scalability Distributed Volume MBps vs RHS Nodesvs RHS Clients Throughput MBps m8 916 624 m32 Distributed Mode Sequential Read Performance Scalability RS Cluster Size nodes vs Throughput MBps Client threads Nodes gt a a SST ED 679 08 ooo o s o 1284 66 2208 96 i i 2388 97 ee e ee E 1248 14 14 1024KB iozonefile transfer size 8GB iozone record size Figure 15 SeqRd Throughput Scales Linearly as Storage Nodes and or RHS Clients are added Random Read Scalability Distributed Volume MBps vs RHS Nodesvs RHS Clients a a _ 2 a L o 2 lt Sequential Write Scalability Distributed Volume MBps vs RHS Nodesvs RHS Clients Throughput MBps m8 816 n24 32 Distributed Mode Sequential Write Performance Scalability RS Cluster Size nodes vs Throughput MBps Client threads Nodes gt a a Se e iozone record size iozone file transfer size 8GB Figure 16 Seqwr Throughput Scales Linearly as Storage Nodes and or RHS Clients are added Distributed Mode Random Read Sa
40. sfer Protocol HTTP is commonly used as the transport layer basis for REST communication See http en wikipedia org wiki Representational State Transfer SAN Storage Area Network is a storage server or appliance that uses block based protocols typically based on SCSI to access files over a fiber channel or TCP IP network See http en wikipedia ord wiki Storage area network Scale out Storage SOS SOS is a usage model for storage that enables an enterprise to grow capacity incrementally as it adds more storage nodes typically as a new server on an IP network The goal of scale out storage is to grow capacity with near linear versus lump sum investment TCO Total Cost of Ownership 20 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage To learn more about deployment of cloud solutions visit www intel com cloudbuilders For more information on Intel Xeon processors see www intel com xeon For more information on Red Hat Storage Products see http www redhat com products storage server For more information on Intel Intelligent Power Node Manager Technology see www intel com technology nodemanager Disclaimers A Intel processor numbers are not a measure of performance Processor numbers differentiate features within each processor family not across different processor families See www intel com products processor_number for details INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECT
41. t hostnames and their respective glusterfs mounts that we created in the previous section when mounting RHS volumes IOZONE_FILESZ is 8GB we use 8GB file transfers as representative workload for our Content Cloud reference architecture testing IOZONE_RECORDSZ is varied between 64KB and 4MB considering record sizes that are powers of 2 This range of record sizes is meant to characterize the effect of record block size in client file requests on IO performance The lOZONE_TEST parameter is varied to cover the Sequential Read Sequential Write and Random Read Write test cases Each test outcome was averaged over 6 test runs to achieve a 90 percent confidence interval Test results are discussed starting in the section Test Results Distributed Volumes Test Preconditions Clear data inodes and dentries on all RHS server nodes before Starting iozone tests in order to reduce caching effects mais OI 3 Shyeccll Ww wii chee eCaclaee ss Test Results Distributed Volumes Throughput MBps vs Request Sz KB 64 128 256 512 1024 2048 4096 eee SeqRd SegqgqWr e eRandRd lt RandWr Figure 14 RHS Distributed Volume 4 nodes 32 RHS clients SeqRd Wr saturate at reqsz gt 64KB Random Rd Wr Scale over all reasonable block sizes up to 4MB and possibly beyond The chart in Figure 14 illustrates the effect of request size kiloBytes variation l1Ozone record size on RHS Dist
42. tion of GlusterFS high availability and fault tolerance Red Hat Storage GlusterFS for the Enterprise Red Hat Storage is an enterprise implementation of GlusterFS that integrates hardened and tested GlusterFS distribution and industry standard Red Hat Enterprise Linux in an easy to consume distribution for rapid seamless deployment of on premise public or hybrid cloud storage Red Hat Storage uses TCP and XFS by default In addition Hadoop UFO support is built in and validated Red Hat Storage Technical Overview Red Hat Storage based on GlusterFS technology is a distributed filesystem built on top of XFS as its local file system GlusterFS takes a layered approach to the file system where features are added removed as per the requirement This modular architecture allows users to create multiple volumes with different characteristics like distribute replicate with differing performance requirements Though GlusterFS is a File System it uses already tried and tested disk file systems like ext3 ext4 xfs etc to store the data Here are few concepts at the core of GlusterFS VOLUME BRICK SERVER NODES A namespace presented as The basic unit of Contain the bricks a POSIX mount point and is storage represented by comprised of bricks an export directory on a server Figure 3 GlusterFS Concepts GlusterFS Brick Brick is the storage filesystem that has been assigned to a GlusterFS volume A brick is GlusterFS bas
43. uaredtidinseeaaniansgaaeane 14 Test Results Distributed Replicated VOIUMES 00 c ccc ccc een eee eR EEE EER EEE HEED EEE EE dea E EE EEE EES 16 Ferlormance Considerado searr ritr rtir nr ANE ra rE EASE IEEE orien spyacieelenass 18 Other DAINOS tO CONSIOET crasc54gcdteen sorora sy sap paves E R T E A O ETS E 19 mo BO O ee a ene eee re een ne eee ee eee ee Te re er eee 19 CONCIUSION a po E T ue ce unnee rues tenes A S E E E ses pease aaee weeseetenee as 19 IG SSaiY one arene a a R ASES EE E A S E S nue vaeceecesaceee 19 Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Executive Summary To meet escalating capacity requirements in the cloud providers are turning to a Scale out storage infrastructure Red Hat Storage based on the GlusterFS technology combines industry standard servers and standard storage components The GlusterFS architectural model basing cloud storage ona set of converged storage servers allows providers to scale resources to boost capacity bandwidth and I O performance as required with minimal time to deployment and a lower total cost of ownership TCO Illustrated in this paper is scale out storage reference architecture to demonstrate a content cloud storage deployment with Red Hat Storage built on top of Intel Xeon processor family Introduction Unstructured data is the fastest growing type of data in the enterprise today and rich media content has continued to
44. up capacity Figure 2 RHS Global Namespace Linear Scalability Intel Cloud Builders Guide Content Cloud Storage with Red Hat Storage Elastic Deployment Agnostic Runs on Standard Hardware Red Hat Storage works on industry standard Intel hardware no purpose built hardware needed It allows IT admins to add remove storage resources on demand without application disruption Scale Out No Metadata Server Red Hat Storage eliminates the need for a separate metadata server and locates file algorithmically using an elastic hash algorithm This no metadata server architecture ensures that there is no central point of failure improves performance and provides linear scalability High Availability Replication and Geo Replication The replication feature in Red Hat Storage provides active active mirroring within a data center while ensuring high availability and fault tolerance in the event of server OS crashes and networking failures This synchronous replication feature also supports automatic self healing with the latest version of the data when the failed servers are recovered or restored Geo replication enables active passive mirroring of data over wide area networks It provides the means to ensure global fault tolerance and business continuity in the face of natural disasters and other data center wide failures Predictable Performance across a Variety of Workloads As a scale out software only system Red Hat Storage was
45. xpense of energy efficiency Balanced Performance Weights optimization toward performance while comserving energy Balanced Pee Weights optimization toward energy comservation with good performance Pouer Optimization is strongly anie energy efficiency even at the expense of performance Figure 12 CPU Power and Performance Policy Chosen Performance Next enter Advanced BIOS menu and choose CPU power and performance policy to be optimized for Performance Red Hat Storage Node RHS2 x install RHS server nodes rhsO1 rhsO2 rhsO3 and rhsO4 were installed with Red Hat Storage 2 0 update4 This RHS release provides a GlusterFS 3 x implementation that is validated and hardened in the Red Hat Enterprise Linux 6 x environment Red Hat Storage comes preinstalled with GlusterFS server software client daemons CIFS NFS UFO services and Linux kernel tuning daemon tuned tuned adm Tuned provides predefined profiles for optimizing the system performance for the following workloads throughput performance enterprise storage and latency storage OS Disk Creation RAID1 In our reference test environment we used mirrored RAID1 volume for OS install RHS 2 0 update 4 The OS volume was created using the Intel Rapid Storage Technology Intel RST BIOS shown in Figure 13 NJ Hay orage ogy enterprise Copyr ight C Z093 12 Intel Corporation SCU Uption eO All Ri
Download Pdf Manuals
Related Search