Home

Administrator`s Manual

1. Authentication Services Power Management Virtual Machine Startup Shutdown Virtual Machine Swapfile Location Security Profile Host Cache Configuration System Resource Allocation Agent VM Settings Advanced Settings Status Completed Completed Completed Initiated by System System MarcFleischmann 8 8 2012 12 51 37 AM Requested Start Ti Start Time 8 8 2012 12 52 01 AM 8 8 2012 12 51 59 AM 8 8 2012 12 52 01 AM 8 8 2012 12 51 59 AM 8 8 2012 12 51 37 AM Hardware View Datastores Devices Processors Datastores Refresh Delete Add Storage RescanAll Memory Identification 7 Status Device Drive Type Capacity Free Type N gt Storage B ss Norma LocalATADisk t Non SSD 367 50GB 31454GB VMFS5 Networking B VWrsiG naciv Y Normal NA NA NA Storage Adapters B Wrs SM inact O Normal N A N A NA Network Adapters Advanced Settings Power Management l 4 m T Software Licensed Features Datastore Details Properties Time Configuration DNS and Routing Name Target or Status contains y 1 Clear X Completed Time 8 8 2012 12 52 04 AM 8 8 2012 12 52 02 AM 8 8 2012 12 52 01 AM Evaluation Mode 46 days remaining Marc Fleischmann Figure 11 vSphere 5 Adding storage to VMFS stores in the Storage View Click Add Storage to bring up the Add Storage dialog check Disk LUN gt Next select the newly added SCSI LUN check VMFS 5 gt
2. 2 then dick Disconnect b Advanced For target properties including configuration of sessions Properties select the target and dick Properties For configuration of devices associated with a target select the target and then dick Devices More about basic iSCSI connections and targets Le Apply Figure 4 Windows 7 Connecting to an iSCSI target 12 2 2 Configuring CHAP Authentication The Connect To Target dialog gt Advanced button brings up a window that allows configuring the SCSI connection with regard to CHAP Authentication and IPSec To configure CHAP Initiator Authentication click the checkbox Enable CHAP log on and enter the target credentials see Section 9 5 2 for the corresponding settings in the targetcli auth group e Enter the target secret matching the targetcli userid and e Optionally adjust the name matching the targetcli password To configure Mutual CHAP Authentication return to the iSCSI Initiator Properties select the Configuration tab and e Click CHAP and enter the Initiator CHAP Secret matching the targetcli mutual_password and e Optionally click Change and set New initiator name matching the targetcli mutual_userid Click the Targets tab gt Connect button gt Advanced button and check the Enable CHAP logon and Perform mutual authentication checkboxes then enter the target credentials see S
3. Spi E SE SY Je l Controller Battery Information Status Not Installed Logical device number 0 Logical device name RAID level Status of logical device Size Stripe unit size Read cache mod Mesi preferret cache setting MaxIQ cache setting Write cache mode Write cache setting Partitioned Pepe lex leng Seu Global Hot Spare Bootable data 5 Optimal 11427830 MB 1024 KB Enabled Enabled Disabled Enabled write back Enabled write back No MES 0 Sal No Copyright O 2015 Datera LIO Admin Manual DATERA Failed stripes 8 Mo Power settings Disabled 3 2 3 LSI Display comprehensive information about the RAID controller including RAID state etc T MegaCli AdpAllinfo a lt controller gt Display the information of one or more logical drives on one or more selected RAID controllers MegaCli LDInfo L lt drive gt a lt controller gt The drive ID and controller ID can be wildcarded by substituting the ID with the ALL qualifier With storage arrays based on LSI RAID controllers Linux also supports remote alert notifications The corresponding alert settings mail server setup and email recipients can be configured through the LSI MSM client To that end launch the MSM client and connect to the IP or FQDN of the Linux array to be managed From the toolbar select Tools gt Configure Alerts The Configure Alerts dialog then appears where the desired RAID cont
4. 3 1 2 Drive identification The Adaptec and LSI RAID CUS always require the controller number to be supplied as a command line parameter The exact numerical value depends on your Linux system array configuration The RAID controller IDs start with 1 on Adaptec controllers and 0 on LSI MegaRAID controllers Identify the drive s to operate on with the following commands respectively Adaptec arcconf IDENTIFY lt controller gt DEVICE lt channel gt lt slot gt LSI MegaCLI PDLocate PhysDrv lt enclosure slot gt a lt controller gt The corresponding drive s can then be identified by the blinking activity LED in their respective ports 3 2 System Management The health of your system is instrumental for the effective SAN operation and Linux includes a number of options to manage and monitor the systems operation 3 2 1 Viewing RAID States The Adaptec and LSI RAID CLIs allow viewing the current status of the RAID configuration Every RAID set is referred to as a logical device and therefore the state of the logical device is the most important information for administrators to monitor in terms of its functionality and performance A healthy RAID set shows its status as optimal to indicate fully operational state with optimal performance Any other status mandates careful investigation by the systems administrator including taking the appropriate action to restore
5. Full Copy XCopy Commands the array to duplicate data in a EXTENDED_COPY LUN Used for Clone and VMotion operations Space Reclamation Allow thin provisioned arrays to clear unused Yes Yes Yes UNMAP VMES space Disabled Table 6 Overview over VAAI Primitives The presence of VMware VAAI and its features can be verified from the VMware ESX 5 CLI as follows esxcli storage core device vaai status get naa 6001405a2e547c17329487b865dla66e VAAI Plugin Name AMS GEES Clone Status SUO Zero Status supported Delete Statuss Unsupported Note Delete is disabled per default see below for more details 11 3 Primitives 11 3 1 ATS ATS Atomic Test and Set is arguably one of the most valuable storage technologies to come out of VMware It enables locking of block storage devices at much finer granularity than with the preceding T10 Persistent Reservations see 12 which can only operate on full LUNs Hence ATS allows more concurrency and thus significantly higher performance for shared LUNs For instance HP reported that it can support six times more VMs per LUN with VAAI than without it Copyright 2015 Datera 58 LIO Admin Manual DATERA ATS uses the T10 COMPARE_AND WRITE command to allow comparing and writing SCSI blocks in one atomic operation NFS doesn t need ATS as locking is a non issue and VM files are shared differently than LUNs are Feature presence can be verified from the VMware ESX
6. The basic layout of the targetcli object tree is as follows oe 9 loyvelssO es gr fileio G filelo storage objecten Woloele as iblock suezage eloyest 1 O pece e psesi stozage oUsject 5 O el ele e ze de storage osject ie er wel maja OS rd mcp storage object JL im WD Sipe G Target hi al o acels Os node acl 11 1 os masecel Luna 0 1 o luns o lia gt 12 Q igesi er target 1 0 e oa Lia Copyright O 2015 Datera 24 LIO Admin Manual NATERA o acile e node scil pisa l e mee Jia 10 1 luns o lua E 0 d portals d e d O IP ajoowie 1d al o alar ox ESTAS ENE ia O aele G nede exe 13m mem Tun On OS Lun 1 0 zer 6 7 Creating a LUN and exporting it Creating a LUN requires the following steps Create a backstore object The backstore object is created by navigating to cd backstores and creating a corresponding object in the specific context Create a Target A base Target can be created by navigating to the corresponding fabric e g cd iscsi or cd qla2xxx and entering the create command For iSCSI targetcli automatically assigns a TPG and TPG tag and for all other fabrics targetcli masks the TPG Create the corresponding LUN The LUN is created and connected to the backstore object Create an iSCSI Network Portal For
7. a0e4all tpgtl set attribute authentication 0 demo mode write protect 0 generate node acls 1 cache dynamic acls 1 Buerg len gale ELIES eeneg 2 mue TI Parameter authentication is now 0 ponamegenicremems almo cei edil 18 Mr v II Peremerer cacas eyaemis acils 18 mow 1 iscsi iqn 20 a0e4all tpgtl cd ZS security and data integrity hazards Don t do this for production setups unless you know Warning Exporting open LUNs with no authentication requirements create a significant exactly what you are doing Please see Section 9 5 for detailed examples on how to configure iSCSI authentication for production setups 5 2 5 Display the Object Hierarchy The resulting object hierarchy looks as follows displayed from the root object EE El Copyright O 2015 Datera 15 LIO Admin Manual DATERA ope EE EE EE 0 Storage OoJect A EE EE EE 1 Storage Object O my disk m dev sdb activated E L EE 0 Storage Object 5 Tal che oda nro ooo o Oo 000 a oo apa OOO OOO O EE EE 0 Storage Object O Tcl MOS c o ten o 0 b 0 0 D 0 0 0 0 d C donan ooo D 0 0 0 0 0 0 ooo oa Dr aa opio o OO 0 9 0 Storage Object EECHER 0 Target EE 1 Target G asia 2003 01 euer laine eesi HSR E LLLE 0004350 0913352021 EL Wee O toee maaana o EE EE enabled EE EE TCI 0 ACL OS AS 200 EE EELER 1 LUN G Joao m iblock my disk dev sdb Or portals aaa EE Porra SA EE e EE EECH OK L
8. e If size is a string the following units can be used e Borno unit for bytes ek K kB KB for kB kilobytes e m M mB MB for MB megabytes e g G gB GB for GB gigabytes e t T tB TB for TB terabytes lt generate_wwn gt is an optional Boolean parameter specifying whether or not a T10 WWN serial should be created for the unit by default yes lt buffered gt is an optional Boolean parameter specifying whether buffered mode is enabled The default is disabled by default i e synchronous I O mode 8 3 2 The delete Command delete lt name gt Recursively delete the storage object with name If there are LUNs using this storage object they will be deleted as well Copyright 2015 Datera 32 LIO Admin Manual NATERA Example delete mystorage Deletes the storage object mystorage and all associated LUNs 8 4 The PSCSI Backstore Context 8 4 1 The create Command create lt name gt lt device gt Create a new PSCSI storage object with name and on the physical SCSI device lt name gt is the name of the PSCSI new storage object lt device gt can either be a full path name to a SCSI device or the SCSI device ID If device is a path name Datera recommends using the dev disk by id hierarchy to ensure consistent naming across reboots and or modifications of the underlying physical SCSI system Specifying device as a SCSI device ID in the traditional H C T L format is not recommended as SCSI IDs
9. e nopin_response_timeout integer Values 0 60 default value 5 The value in seconds to wait before failing a path due to a non received NoplN response e nopin_timeout integer Values 0 60 default value 5 The value in seconds to wait before sending a target generated NopIN e random_datain_pdu_offsets bool Values 0 1 default value O disabled Boolean value per RFC 3720 see 6 to signal randomization of DatalN PDU offsets when using DataPDUInOrder No e random datain seq offsets bool gt Values 0 1 default value O disabled Boolean value per RFC 3720 see 6 to signal randomization of DatalN Sequence offsets when using DataSequencelnOrder No e random r2t offsets bool Values 0 1 default value O disabled Boolean value per RFC 3720 see 6 to signal randomization of Ready to Transmit R2T offsets when using MaxOutstaningR2T gt 1 8 14 The Mapped LUNs Context There are no command or parameter groups that apply to this context 8 15 The Portals Context iSCSI This portals context is specific to iSCSI It has no equivalent in any other fabric In iSCSI there are currently no command or parameter groups that apply to it 8 16 The portals Context iSCSI This portals context is specific to iSCSI It has no equivalent in any other fabric 8 16 1 The create Command create lt ip_address gt lt ip_port gt Create an iSCSI Network Portal with specified ip ad
10. ib SEPE oconocsopoooccooooocoooonn cooooocoococooncgonoopooooooocn nap 0 Target Ee MS A ARA a O A AAA GLO OLET a AAA DAS 2 Targets o icm 2003 01 eet laws SAMO 0004381 Seen T Mel povocoooo LME SOS stp ost a cats pelo rada EE enabled Gk COILS Ga oo 5095S SH dH DO ec odo oco oco CODD D OOO OC COON e e BO OOS SRA es 1 ACL ee iiem L t Con Ee HLT Ie dS o onuocoooocooooc 2 Mapped LUNs o mapped beet lunO rw O meysoec IWMI 555505oocoooocoocooooooopooooococcoooopbcocopoo9 lunl rw Qr IMME co000s000c090000n0090000000009000009090900090000090000909990 00 2 LUNs Copyright O 2015 Datera 51 LIO Admin Manual DATERA EE iblock block backend dev sdb Mos Deele Eileigo tileig lvacikend weree Elite OF EES e EE EE EE EE oooO EEN 0 O00 3L Portal Or EE OK O igm 200S5 01 org linm isesi Sand 3359664 8 un H I coooocoso LL 1 Sz GI I ano oo CO OUO EE enabled oU odi IE UU 0 ACL fk AMOS Qo 666600445 e DEE OREO 0 0 0 O0 OUO C SOOO Oe 67818 E 1 LUN O ME SERT T SUN NE pscsi pscsi backend dev sr0 TAE UL oopperaa oooO ooo oboe oaths 6 6 C GO ooo 3L Portal EE OK Lee ee 0 Target DS O EE EE EE 10 Target iscsi gt cd ZS 9 8 Persistence The target configuration without the user data contained on the LUNs can be persisted across reboots by invoking saveconfig from the root context gt saveconfig WA yo Th Ar S Qs zu a u a u a u a
11. 300 default value O Value of legacy tasks timeout in seconds Removed in v4 1 target code Copyright O 2015 Datera 34 LIO Admin Manual MATERA 8 7 The Fabric Modules Context 8 7 1 The create Command create target lt wwn gt Creates a new target lt wwn gt is an optional parameter that specifies the T10 WWN unique serial number to be used for the target Its format depends on the transport s supported by the fabric module If the wwn is omitted a target will be created using either a randomly generated WWN of the correct type or the first unused WWN in the list of possible WWNs if one is available If WWNs are constrained to a list i e for hardware targets addresses and all WWNs are in use the target creation will fail The info command provides more information about WWN types and values 8 7 2 The delete Command delete wwn Recursively delete the target with the specified wwn including all storage objects under it 8 7 3 The info Command info Display information about the fabric module notably the supported transports s and accepted WWN format s along its supported features 8 7 4 The version Command version Display the target fabric module version 8 7 5 The discovery auth Config Group Note This group is only active for the iSCSI fabric module It appears under the global iSCSI context in sys kernel config target iscsi discovery_auth Discovery authentication is typically disab
12. 8 8 1 The create Command create tpg tag iscsi Create a new Endpoint within the current target tpg tag iscsi is an optional iSCSI parameter that must be a positive integer If omitted the next available ordinal number will be used as the TPG tag 8 8 2 The delete Command delete tpg tag iscsi Delete the TPG with the specified TPG tag including all storage objects under it tpg tag iscsi specifies the iSCSI TPG to be deleted It must be positive integer matching an existing TPG tag 8 9 The TPG Context iSCSI The TPG context is specific to iSCSI It has no equivalent in other fabrics 8 9 1 The disable Command disable Disable the TPG 8 9 2 The enable Command enable Enable the TPG 8 9 3 The attribute Config Group e authentication lt bool gt Values 0 1 default value 1 enabled Control for enforcing iSCSI authentication for SCSI Target portal group context Copyright 2015 Datera 36 LIO Admin Manual MATERA e cache dynamic acls bool Values 0 1 default value O disabled Control for caching dynamically generated initiator NodeACLs from generate node acls 1 e crc32c x86 offload bool Values 0 1 default value 1 enabled Control for enabling SSE v4 2 CRC32C offload on supported x86 processors CRC32C offload will automatically be used if the underlying processor supports it which is true for Intel processors starting from the Nehalem microarchitecture Re
13. Taking you back to iscsi iqn 2003 01 org linux iscsi targetcli demo x8664 sn 05135a0e4all tpgtl isesi icm Sterkens 5 5 5 Define Access Control Configure access control Typically this involves setting up ACLs with individual login information for each initiator For a simple demo setup allow access to all initiators without any authentication Copyright O 2015 Datera 20 LIO Admin Manual MATERA iscsi iqn 20 a0e4all tpgtl set attribute authentication 0 demo mode write protect 0 generate node acls 1 cache dynamic acls 1 Parameter Cemo moce write protect is mow TT Parameter authentication is now 0 Boisameqenienemaliselmoceaa titer meng E uereg hel mt ele 2 eeng E iscsi iqn 20 a0e4all tpgtl cd ZS Warning Exporting open LUNs with no authentication requirements create a significant security and data integrity hazards Don t do this for production setups unless you know exactly what you are doing 5 5 6 Display the Object Hierarchy The resulting object hierarchy looks as follows displayed from the root object gt 1s O M C PRINTER EESTI SS IDEICIASUONSS 660464600 Q O 0 0 0 D 0 0 0 6 C 0 0 0 0 O G OO OO 0 0 EELER EE o D O ee ORO oO yb erar Peta d dde noa POOR ORPHEO NER 0 Storage Object o GOTO E EE 1 Storage Object l o my disk seese EN a E Ee EIU dev sdb activated NS TU AAA O RARO EE E 0 Storage Object O wel Cl PL C T 0 Storage Ob
14. ZN RNING e cur e you CCess CCess CCess king bac Fully upda king bac Fully upda king bac Fully upda king bac Saving eumd limux 1sesi org Current COE d pne er 1 01 ur boot settings ren E targe s re Type kup o KU o kup o kup o qla2xxx ConfigFS with timestamp bedadetaultecontige etueytasgetu ia2s st as loggoecik Contigis wita tiuessteawgps 2011 13 15 2149556574752 sed deteult contig ete terget loopback sta sayt Comticis witha timesteamos 2011 11 15 21 Eed dlefemilie contig ete target srpt starts LOTE ge 5 a 15 z etc targe nerat alas king bac gl It ata 5 14 192 o po KU E larget Co AMAS T A Gener ec Target Core Moc eos acia pea aclaro 2011 11 15 218419858 3974792 in to disk will overwrite Comrigureiciom willl become the defewlt boo yes yes E COME SD Lt 294149 ss 9747192 Eo Sila i s Sim pA 8 5181 c o p a h Comics wita tinmesteamos 2i i1 15 2149 56 574752 t config etc target backup lio backup 2011 11 sh re Mod ConfigFS with timestamp 2011 11 Success wlly vpcated dstfamlte contig ste terget lio mat sia Successitully updated detewult contig ete target teu startsh ZS Note Without saveconfig the target configuration will be lost upon rebooting or unloading the target service as the target configuration will be rolled back to the last saved one 9 9 Navigation and Auto Completi
15. iscsi IQN tpgt1 Associate an IP address portals create IP address In iscsi IQN tpgt1 Export the LUN my disk luns create backstores iblock my disk In iscsi lt IQN gt tpgt1 Enable Demo Mode set attribute authentication 0 Beware demo mode write protect O generate node acls 1 cache dynamic acls 1 Table 2 Setting up an iSCSI target 5 2 2 Instantiate an iSCSI Target Instantiate an iSCSI target on the backstore device to form a Target Portal Group TPG backstores iblock my disk iscsi create Created target ign 2003 01 org linux iscsi targetcli demo x8664 sn 05135a0e4all selected WIE oer La SUECESSTVUL IN created Wee IL Entering new node iscsi iqn 2003 01 0rg linux iscsi targetcli demo x8664 sn 05135a0e4a11 tpgt1 iscsi iqn 20 a0elal1 tpgt1 gt Targetcli creates the TPG automatically assigns the next default TPG tag 1 and changes the working context to the resulting tagged TPG Assign an IP address here IPv4 192 168 1 139 to the TPG to make it accessible to iSCSI initiators iscsi iqn 20 a0e4all tpgtl portals create 192 168 1 139 EE cefeult 1 port 3260 Copyright 2015 Datera 14 LIO Admin Manual MATERA Successfully created network portal 192 168 1 139 3260 Entering new node iscsi iqn 2003 01 0rg linux iscsi targetcli demo x8664 sn 05135a0e4a11 tpgt1 portals 192 168 1 139 3260 JALSOSL strega 20 5 oo S841 1393 32 607 Targetcli add
16. ls my first target O EpgEL resse sas a are atea aaa apodaca us SIS EE No ida palacete le aca sio adas enabled O Sila y 0000 ado oooO O EEN EELER 3L CIL cs abigo dE SI 1L 5 Com ml eae ots od te abm ied LIS 5 6 6 o 0 9 9 o 0 0 0 0 0 0 0 0 0 0 6 6 2 Mapped LUNs IAS lum 500050o0ooocnocopoooooocooocoonooooocooooocooonon lunO rw G mejjeo IWMI 55550o0cocoooocanoooboocoocoooca aooooocoooooococan lunl rw OS AUIS as a DINNER SD EU EO da 2 LUNs as MWD ooooocooccooocooooncooocoocooc coooo iblock block backend dev sdb o unl e rt E Nd Ee a E leia backen usr sre iileio EE 3L Portal AS AS A E E iren o O 66 OK gt cd my first target iscsi iqn 20 196ff4e tpgt1 gt Copyright O 2015 Datera 54 LIO Admin Manual NATERA 10 RTSlib Storage Management Library and API 10 1 iSCSI 10 1 1 Setup Script The following Python code illustrates how to setup a basic iSCSI target and export a mapped LUN usr bin python iSCSI setup script example with RTSlib Erom sl amore Setup an IBLOCK backstore backstore IBlockBackstore 3 mode create AS so IBlockStorageObject backstore sdb dev sdb gen_wwn True except backstore delete raise Create an iSCSI target endpoint using an iSCSI ION selgicals Ieloracmocule aisesi y target target febrie lem 2003 01 cig JL3aapiss d exea Eu sm celia 300 tpg TPG target 1 Setup a network portal
17. 4 on logical drive 1 in RAID controller O as setup above MegaCli LDRecon Start r5 Add PhysDrv 0 4 L1 a0 3 6 Failure Management A soft error doesn t necessarily indicate a hard drive failure For instance after years of operation some drives may occasionally not respond fast enough so the RAID controller flags them as failed despite the fact that they are still fully operational In such cases a rescan of the SCSI bus may re add drives back to the array without the need of any drive replacements However soft errors are a frequent characteristic of aging devices and in many cases are indicative of subsequent hard drive failures so Datera highly recommends replacing drives that exhibit soft errors In any event failed drives must promptly be replaced with a spare drive Depending on the underlying RAID level a degraded RAID set might not be fault tolerant during RAID reconstruction e g RAID5 3 6 1 Adaptec Rescan the SCSI bus arcconf RESCAN lt controller gt A drive failure condition can be identified as follows arcconf GETCONFIG 1 PD egrep Device State gt Reported Location Reported Channel S M A R T Warnings After the failed drive has been replaced with a standby add a new dedicated hot spare drive arcconf SETSTATE lt controller gt DEVICE lt channel gt lt port gt HSP LOGICALDRIVE lt drive gt Copyright 2015 Datera 9 LIO Admin Manual
18. 6 7 8 9 10 11 12 13 14 15 16 17 18 Adaptec RAID controller Command line utility User s Guide Adaptec 2008 http download adaptec com pdfs user guides CLI v6 10 Users Guide pdf Adaptec Storage Manager User s Guide Adaptec 2008 http download adaptec com pdfs user guides asm v6 10 users guide for das pdf Dafna Sheinwald Internet Small Computer Interface iSCSI Cyclic Redundancy Check CRC Checksum Considerations RFC 3385 September 2002 http www ietf org rfc rfc3385 txt Ethereal web site http www ethereal com Glen Turner Remote Serial Console HOWTO Australian Academic and Research Network http www tldp org HOWTO Remote Serial Console HOWTO Julian Satran Internet Small Computer Interface iSCSI RFC 3720 IETF August 2004 http www ietf org rfc rfc3720 txt LSI Embedded MegaRAID Software User s Guide Version 2 0 LSI November 2006 http www lsi com downloads Public Storage9620Products Internal9620RAID embedded mr sw ug pdf LSI MegaRAID SAS Software User s Guide Version 2 0 LSI June 2007 http techpubs sgi com library manuals 0000 860 0488 001 pdf 860 0488 001 pdf Microsoft iSCSI Initiator Windows Server 2008 R2 and Windows 7 Microsoft March 2012 http www microsoft com en us download confirmation aspx id 6408 Oracle VM Virtual Box User Manual Oracle 2011 http www virtualbox org manual Datera Error Recovery Level
19. All PO Device Type WWN Meneiry iSCSI Software Adapter FN G wmhba34 SCSI ign 1998 01 com vmware localhost 34f4895a Networking ICH10 2 port SATA IDE Controller Ba Q vmhbat Block SCSI G vmhba33 Block SCSI Network Adapters ICH10 4 port SATA IDE Controller Mivancell Settings G vmhbao Block SCSI Power Management Q vmhba32 Block SCSI Software Licensed Features Time Configuration DNS and Routing Authentication Services Details Power Management hba34 Properties Virtual Machine Startup Shutdown Moda ISCSI Software Adapter Virtual Machine Swapfile Location iSCSI Name ign 1998 01 com vmware localhost 34f4895a Security Profile iSCSI Alias Host Cache Configuration Connected Targets 6 Devices 6 Paths 6 System Resource Allocation View Devices Paths Agent VM Settings Advanced Settings Name Runtime Name Operational State LUN LIO ORG SC naa 600140573d94f8e531d4d 1ab5c8a72ef 72 L0 Unmounted H L C naa 60014050e4485b 9bdc8 0 E 0 o U Recent Tasks Name Target or Status contains lt Clear Name Target Status Initiated by Requested Start Ti Start Time Completed Time Z Rescanvmrs H 192 168 1 28 Completed MarcFleischmann 8 7 2012 11 40 05PM 8 7 2012 11 40 05 PM 8 7 2012 11 40 07 PM 2 Rescan all HBAs H 192 168 1 28 Completed MarcFleischmann 8 7 2012 11 40 02 PM 8 7 2012 11 40 03 PM 8 7 2012 11 40 05 PM MarcFleischmann 8 7 2012 11 39 21 PM 8 7 2012 11 39 21 PM 8 7 2012 11
20. Choose the version with the Authenticode Signature Reboot o Other drivers MT25408 C Program Files OFED drivers lOU SRP miniport C Program Files OFED drivers SRP Reboot Windows Server 2012 into Normal Mode Copyright 2015 Datera 68 LIO Admin Manual DATERA 12 4 VMware vSphere 5 iSCSI Initiator 12 4 1 Connecting to Targets Start the VMware vSphere Client and connect to the VMware vCenter Server or directly to the to the ESX host In the VMware vSphere Client resource menu on the left side select the datacenter cluster and ESX host on which you like to connect the new data store Go to the Configuration tab gt Storage Adapter right click iSCSI Software Adapter and Properties m ag File Edit View Inventory Administration Plug ins Help F wan a Tea O EI A tome pa inventory pf Hosts and Custers EH Search Inventory El Gp WIN KF99MLLVB9Q E Ey VM Workbench El A 192 168 1 28 192 168 1 28 VMware ESXi 5 0 0 469512 Evaluation 53 days remaining Getting Started Summary Virtual Machnes ResourceAllocation Performance Configuration Tasks amp Events Alarms Permissions Maps Storage Views E Y Detach SCSILUN RH 192 168 1 28 Completed Z Tasks Alarms Figure 8 vSphere 5 a eso Hardware Storage Adapters Add Remove Refresh Rescan
21. Clusters Pu Sees la Search Inventory 19 S Gi WIN KFSSMLLVB9Q El Ey VM Workbench amp A 192 168 1 28 192 168 1 28 VMware ESXi 5 0 0 469512 Evaluation 53 days remaining Win SSD EE E ETA eN Configuration WE ARE ECCL UTE ERE ET ERES GTC ERIT R 192 168 130 Hardware View Datastores Devices l l Processors Datastores Refresh Delete Add Storage Rescan All Memory Identification Device Drive Type Capacity Free Type Storage BH sso Norma Local ATA Disk t Non SSD 367 50 GB 314 54 GB VMFSS Networking B VMFS FIO1 Normal LIO ORG iSCSI Di Non SSD 14975 GB 148 80 GB VMFSS Storage Adapters 8 Norr ORG iSCSI D D N A NA NA Network Adapters B8 N A N A NA Advanced Settings Power Management m Software Licensed Features Datastore Details Properties Time Configuration VMFS FIOL 149 75GB Capacity DNS and Routing Location vmfs volumes 50228d13 15b96ab8 7cef 001e6726c4bd Hardware Acceleration Supported 971 00MB W Used Authentication Services 148 80GB D Free narii Refresh Storage seng System Storage Capability N A Virtual Machine Startu doun dide User defined Storage Capabiity N A Virtual Machine Swapfile Location Security Profile Path Selection d Host Cache Configuration Most Recently Us 2 3 9 Satiaki pm Volume Label VMFS FIO1 LIO ORGiSCSIDisk naa 6 149 88 GB Disable ys esource Allocation Datasto
22. Linux Target wiki 2013 http linux iscsi org wiki Error Recovery Level Datera Persistent Reservations Linux Target wiki 2013 http linux iscsi org wiki Persistent_Reservations Datera vStorage APIs for Array Integration Linux Target wiki 2013 http linux iscsi org wiki VAAI SuSe Zypper Usage Version 11 1 27 November 2009 http old en opensuse org Zypper Usage 11 1 Vivek Goyal Documentation for Kdump November 2011 http www kernel org doc Documentation kdump kdump txt VMware Disable Space Reclamation In ESXi and vCenter Server 5 Documentation April 2012 http pubs vmware com vsphere 50 index jsp topic 2Fcom vmware vsphere storage doc_50 2FGUID EB15AF57 OAD2 414D BE5D 3AAF4623133D html William Allen Simpson PPP Challenge Handshake Protocol CHAP RFC 1994 August 1996 http www ietf org rfc rfc1994 txt Wireshark web site Riverbed Technology http www wireshark or Copyright 2015 Datera 76 DATERA THE CLOUD IS THE COMPUTER Datera Inc 2570 W El Camino Real Suite 380 Mountain View CA 94040 Copyright O 2015 Datera Inc All rights reserved Datera LIO and the Datera logo are trademarks of Datera Inc which may be registered in some jurisdictions
23. Next gt Next and enter a datastore name e g VMFS FIO1 Click Next and the final Add Storage dialog summarizes the information for the newly added LUN Click Finish to complete its setup d eks r Po NAAA e a Select Storage Type Ready to Complete Specify if you want to format a new volume or use a shared folder over the network Review the disk layout and dick Finish to add storage Storage Type Disk LUN Disk layout M Ready to Complete o he LIO ORG iSCSI Disk naa 60014 Non SSD 149 89 GB 0 Formatting Network File System vmfs devices disks naa 60014055ff891d0c5b4260266942400 Ready to Complete Choose this option if you want to create a Network File System 0 a a GPT Primary Partitions Adding a datastore on Fibre Channel or iSCSI will add this datastore to all hosts that have access f VMFS LIO ORG SCSI Disk naa macn to the storage media File system Properties Datastore name VMFS FIO1 Formatting File system vmfs 5 Block size 1MB Maximum fie size 2 00T8 es as mes c el ET Al Al 4 Figure 12 vSphere 5 Adding LUNs to VMFS stores Copyright O 2015 Datera 71 LIO Admin Manual DATERA The new VMFS store VMFS FIO1 now shows up in the Storage view of the vSphere Client File Edit View Inventory Administration Plug ins Help gag a e sx a Home p ei Inventory p 8j Hosts and
24. a EE e o 0 Storage Object O ZA MED o5655ocoo0c00000000c0050500000000000000509090900020 0 Storage Object ies BO Ie aora erpnpnooood Ee 1 Target C OxDOOOUCOOOOOOOCOOOOO2ZSSOSOOOSSeeCl oooonorococoonocoosonapronoo enabled O LS 5500oo00000 22400000009090000000040090900000000009 0999000909098 6 600 1 ACL G eat DCH Elie sss2soocococcoococcoc 1 Mapped LUN AMAS lun i5 50550oo0ooon5c0c00000090co0050090000000000690005 lunO rw O Jibs o3 ce c opio o oo Co OO ORR AED DO OC OOOO OO RAH EEN 1 LUN O EE iblock my disk dev sdb DS EE T 0 Target NN NN EE a OO TT 0 Target oco EE 10 Target Copyright O 2015 Datera 57 LIO Admin Manual DATERA 11 VMware VAAI 11 1 Overview VMware introduced the vStorage APIs for Array Integration VAAI in VMware vSphere 4 1 with a plugin and provided native VAAI support with VMware vSphere 5 VAAI significantly enhances the integration of storage and servers by enabling seamless offload of locking and block operations onto the storage array LIO provides native VAAI support for VMware vSphere 5 see also 13 11 2 Features The LIO Linux SCSI Target supports the following VAAI functions T10 Primitive Description Atomic Hardware Assisted Enables granular locking of block storage Test amp Set Locking devices accelerating performance ATS COMPARE AND WRITE Block Zeroing Communication mechanism for thin pro WRITE_SAME visioning arrays Used when creating VMDKs
25. and target negotiate the minimum time in seconds to wait before attempting an explicit implicit logout or an active task reassignment after an unexpected connection termination or a connection reset 0 indicates that logout or active task reassignment can be attempted immediately ErrorRecoveryLevel lt integer gt Values 0 2 default value O Copyright O 2015 Datera 38 LIO Admin Manual NATERA The Initiator and target negotiate the supported recovery level Recovery levels represent a combination of recovery capabilities Each recovery level includes all the capabilities of the lower recovery levels and adds some new ones to them see also 11 e FirstBurstLength integer Values 512 2424 1 not to exceed MaxBurstLength default value 65353 The initiator and target negotiate the maximum amount in bytes of unsolicited data an SCSI initiator may send to the target during the execution of a single SCSI command This covers the immediate data if any and the sequence of unsolicited Data Out PDUs if any that follow the command e HeaderDigest enum Values CRC32C None default value None Used to control the HeaderDigest usage by the SCSI target portal group endpoint RFC 3385 provides a discussion on the selection criteria for the CRC see 3 e FMarkint integer Values 1 65535 default value 2048 Used to set the interval for the target to initiator markers on the connection e FMar
26. can vary over time 8 4 2 The delete Command delete lt name gt Recursively delete the storage object with name If there are LUNs using this storage object they will be deleted as well Example delete mystorage Deletes the storage object mystorage and all associated LUNs 8 5 The RDDR Backstore Context 8 5 1 The create Command create name size lt generate_wwn gt Create a new RDDR direct memory mapped ramdisk storage object with name lt name gt is the name of the RDDR new storage object lt size gt specifies the size of the ramdisk The size syntax is e Ifsize is an integer it represents a number of bytes e If size is a string the following units can be used e Borno unit for bytes e k K kB KB for kB kilobytes e m M mB MB for MB megabytes e g G gB GB for GB gigabytes e t T tB TB for TB terabytes lt generate_wwn gt is an optional Boolean parameter that specifies whether or not a T10 WWN serial number should be generated for the LUN The default is true Copyright 2015 Datera 33 LIO Admin Manual MATERA 8 5 2 The delete Command delete lt name gt Recursively delete the storage object with name If there are LUNs using this storage object they will be deleted as well Example delete mystorage Deletes the storage object mystorage and all associated LUNs 8 6 The Storage Objects Context 8 6 1 The version Command version Display the version of the curre
27. cdetewlt WP port 3260 Sucess S created netverk portel 192 166 i 13955260 Entering new node iscsi iqn 2003 01 org linux iscsi targetcli demo x8664 sn 05135a0e4a11 tpgt1 portals 192 168 1 139 3260 isecsi icm 20 gt SG 11 1393 3200 gt Targetcli adds the SER Network Portal and automatically changes the working context to it 5 5 3 Enable SER Enable the SER protocol for the TPG iscsi iqn 20 68 1 139 3260 iser enable iger MOO Sra Eent has ise emere clie fisesa Zem EE alo 1 998 Sege Le A ENEE OK iser enabled Vitiis Ec OU o 681 139 5260 gt cel lt Taking you back to asesal iea 2003 01 Ora o linus iscsi targeteli demo x8664 sn 05135a0e4a11 tpgt1 iscsi iqn 20 a0ela11 tpgt1 gt 5 5 4 Export LUNs via SER Declare a LUN for the backstore device to form an Endpoint a valid network storage object iscsi iqn 20 a0e4all tpgtl luns create backstores iblock my disk Selected LUN 0 Successfully created LUN 0 Entering new node iscsi iqn 2003 01 0rg linux iscsi targetcli demo x8664 sn 05135a0e4a11 tpgt1 luns lun0 lisesi iem 20 5 o orte dL Mosa Lumos Targetcli automatically assigns the default ID O to the LUN and then changes the working context to the Endpoint Now the target is created and exports the local device dev sdb as iSER LUN O Return to the underlying TPG as no attributes need to be set or modified for standard LUNs iscsi iqn 20 gtl luns lun0 ed lt
28. counters in esxtop are Counter Name Description DEVICE Devices that support VAAI LUNs on a supported storage system are listed by their NAA ID You can get the NAA ID for a datastore from the datastore properties in vCenter the Storage Details SAN view in Virtual Storage Console or using the vmkfstools P vmfs volumes lt datastore gt command LIO LUNs start with naa 6001405 Note Devices or datastores other than LUNs on an external storage system such as CD ROM internal disks which may be physical disks or LUNs on internal RAID controllers and NFS datastores are listed but have all zeroes for VAAI counters Copyright 2015 Datera 62 LIO Admin Manual DATERA Table 8 VAAI statistics available in VMware ESX Counters that count operations do not return to zero unless the server is rebooted Throughput counters are zero when no commands of the corresponding primitive are in progress Clones between VMFS datastores and Storage VMotion operations that use VAAI increment clone read for one LUN and clone write for another LUN In any case the total for clone read and clone write columns should be equal 11 6 Best Practices For VAAI Clone operations ESX can increase or decrease the size of the transferred data chunks to adjust to the available network bandwidth From the ESX 5 shell the procedure is as follows Check the current data transfer chunk size for VAAI Clone operations esxcfg advcfg g Dat
29. e lu Ell BEE 29 7 8 The refresh Command iii ie 29 7 9 heset Commnalna acia ido iia 29 4 10 The Status COMM aN DE 30 7 11 Nariable TYPOS EG 30 7 12 The Global Contig GrOUP dida iaa 30 a A E O O 32 8 1 The Root Y Context ect niece tcr nietos da ticas 32 8 2 The backstores Context ceti tr ici cda aa bp E 32 8 3 The FILEIO Backstore ContexXt eet ee etcetera oee seus e een oce a Ee aay raai 32 8 4 The PSCSI Backstore Context 33 8 5 The RDDR Backstore Context 33 8 6 Th Storage Objects Context ctetu did 34 8 7 The Fabric Modules Coney aada orae daa aec ss debs REENEN ee EEN 35 8 8 The Target Context ote niente diia 36 8 9 The TPG Context iSCSI 52 5 eerte tree eege iet ete ve tvm eege 36 8 10 The LUNs Context E 40 8 11 The Juns Context 41 8 12 The Node ges Context crit tee ente er aaia ANO deR ccassisiedesaveiceadeisiedeerites 41 8 13 The Node ACLS Context watevcsctecccssstcrencsscaiccdengcleczentesscdsncetencsncdteceaadsbecgandesecgaatabescassviccsaaiebedsacverecs 41 8 14 The Mapped LUNs Context 43 8 15 The Portals Context SCS idad 43 8 16 The portals Context iSCSI ssl 43 9 Targetcli En E 44 OL Statt p WE 44 9 2 Backstores usen eerta I eduauccedeauieccssauicecessacce ccsiadieceruniec caasslecesavicedsiaiins 44 ECH GC Ree ee ln EE 45 Copyright O 2015 Datera iii LIO Admin Manual MATERA 10 11 12 13 14 9 4 iSCSI Network Portals deceaduceceeausudeaceaton
30. in the iSCSI TPG The IP address must already be active on the system portal NecworkePorcal use YMI9Z 1683 1 128 SO6QU Export LUN 0 via the so StorageObject class Jean eer Lo 0 So my lua Setup the INGCISACI Eo am 1SCSIL imitiator and create Mepaeclun 0 node eel tpg Node acl icm 2003 01 geet linuz isesi o y 004 8 Sid Ele Gle te mal 3 el mapped lun mocks ecl mapped Lua 0 0 False 10 1 2 Object Tree The resulting object tree looks as follows cM UEM DASS T CTI BER ELEME rc LORS GoracemObsicet ee HCHO EE Ho EE E ER 1 Storage Object EI EE dev sdb activated I EE EE 0 Storage Object a ono DODO DD OOD DODD DDD DDD Oooo o 0 Storage Object I UUE O ooo ooo EE 0 Storage Object DAS ad EE 1 Target gr iem 2003 01 eet lia Oslo 604 8 sa elSxeliSle ele ooopocoocooono i WEE O O EE enabled ES EELER 06 0 0 EE o 0 0 EE EE 1 ACL O iiem 2003501 org linuz isesi y EE emm elocclsrto jad LL Merge IU mapped lum 5 5055o0ooooccooccocoooocooooocoocooooocoooooo lunO rw A Suel 8 OS SS ton eto d e GOAL ener Oro BD EE 1 LUN Omar te sieve cue er oe aceras sath tae UE ao eee iblock sdb dev sdb OS portals Nep IP E 3L Postal SS al POGOe CODY oC OO OIM PSqdaect dte ms OK Copyright O 2015 Datera 55 LIO Admin Manual NATERA 10 2 Fibre Channel 10 2 1 Setup Script The following Python code illustrates how to setup a basic Fibre Channel target and expo
31. m node T iqn 2003 01 org linux iscsi san01 x8664 sn 35ee770c82fb p 192 168 1 59 At this point all exported devices are visible in the kernel message ring buffer via dmesg or can be checked via fdisk Copyright 2015 Datera 72 DATERA LIO Admin Manual 12 6 VirtualBox iSCSI Initiator Virtualbox has an integrated iSCSI initiator which be used to access exported LUNS natively without the host needing to be configured at all For more information on how to register iSCSI based LUNs please use the VBoxManage storageattach command as documented in the Oracle VM VirtualBox User s Manual Section 8 VBoxManage StorageAttach see 10 Copyright 2015 Datera 73 LIO Admin Manual MATERA 13 Glossary Backstore A physical storage object that provides the actual storage underlying an iSCSI Endpoint CDB Command Descriptor Block The standard format for SCSI commands CDBs are commonly 6 10 or 12 bytes long though they can be 16 bytes or of variable length CHAP Challenge Handshake Authentication Protocol An authentication technique for confirming the identity of one computer to another Described in RFC 1994 see 17 CID Connection Identifier A 16 bit integer generated by the Initiator that uniquely identifies a connection between two iSCSI devices This integer is presented during the login phase Endpoint The combination of a Target name with an explicit or masked TPG IQN WWN T
32. oes ono EE OOS 0 ACL O LUM AER EE AE OAA AD 000 EE 6 LUNs Eet EE iblock ib DBData01 dev vg ssd lv DBData01 A IE E iblock ib DBData02 dev vg ssd lv DBData02 et c e iblock ib DBData03 dev vg ssd lv DBData03 or H ee iblock ib DBData04 dev vg_ssd lv_DBData04 o lunA e ec eC iblock ib DBData05 dev vg ssd lv DBData05 os aoopopooococcos iblock ib DBData06 dev vg ssd lv DBData06 Om portals ec m TEST 4 Portals ext US CSO o Pe voc a o IM ab O UE ce OK EE EE OK EE OK UN AA SS ESI IG M Meet as ep oa E OK gr icin 2003 01 ore siameses SAMO x8664 89S 7056295986131 opoco0o000 LL WEE O tocco ee EE enabled O BCLS iomoooo000000000d4dfHd40D0000oo0o0090800011T 000 D00002900044 0 o DE 0 ACL Oc RTS E E e E e EEEE E E ET EE E E A O S LS Dis ASA e iblock ib DBVotel dev vg_ssd lv_DBVotel MA ncaa ur wae ous iblock ib DBVote2 dev vg_ssd lv_DBVote2 o Jan oroooeoooccocooncoo iblock ib DBVote3 dev vg ssd lv DBVote3 EIER 4 Tess G AN A A EE EE OK EE EE OK CS HOR H 1 E E Te Nee EE CC C STE ee S OK AAA SAO cuo o n RUM Sm LE Mcd UCM ERI EL RUE OK Qr ausum o 2003 01 Qro lbs SeinOl 5x64 8 Sa ETS 956192 VO ooococnac 1 TEE O Ane cannon eoooo bone EE OOO O OA enabled O GELS 5 55ooccooososooooO0900000090090000000200000d 0000900200500 6600 0 ACL GH AMOS osc ooo oouHn ebb bab E O uot mmsu6 dm 4 LUNs I ere iblock ib VSAL dev vg hdd lv VSAL Mos undue s estt iblock ib VSALMeta dev vg hdd lv VS
33. password initiator secret for the initiator iscsi iqn 20 70c82fb tpgtl cd acls iqn 1991 05 com microsoft ibm t410s iscsi iqn 20 oft ibm t410s gt set auth userid iqn 1991 05 com microsoft ibm t410s password mytargetsecret mutual userid iqn 2003 01 0rg linux iscsi san01 x8664 sn 35ee770c82fb mutual password mymutualsecret Parameter userid is now iqn 1991 05 com microsoft ibm t410s Parameter password is now mytargetsecret Parameter mutual password is now mymutualsecret Parameter mulrualusericd ts mow alone 20080 o gibts iscsi san01 x8664 sn 35ee770c82fb iscsi iqn 20 oft ibm t410s gt get auth AUTH CONFIG GROUP mutual password mymutualsecret mr ee Ak emm 200301 eer Liam abre at 5 santil el 8 sin SSeS 7 10 52 To password mytargetsecret userid iqn 1991 05 com microsoft ibm t410s iscsi iqn 20 oft ibm t410s gt cd iscsi iscsi gt Both the SCSI initiator and target Endpoints are ready for secure logins Note The Microsoft iSCSI Initiator uses its ION for the default login name It requires the password length to be between 12 bytes 96 bits and 16 bytes 128 bits and it rejects passwords that are too simple The Microsoft Windows iSCSI initiator allow changing the default login name through the Targets tab gt Connect button gt Connect To Target dialog gt Advanced button gt Advanced Settings dialog gt Enable CHAP log on checkbox The resulting two iSCSI targets one with ACLs for Mutu
34. the array back to its 100 fully functional status A degraded logical device is still operational but this state indicates that an SSD or HDD failed the logical RAID is not operating at full performance and reliability and requires attention Copyright 2015 Datera 5 LIO Admin Manual DATERA Note With failed hard drives the write back cache of RAID controllers is significantly less effective therefore causing additional RAID performance degradation 3 2 2 Adaptec Display comprehensive information about the RAID controller including RAID state etc arcconf GETCONFIG lt controller gt LD Controller information Contro Mieri rattus Channel description comercia Modei Installed memory Copyback Background consistency check Automatic Failover Global task priority Performance Mode Stayawake period Sector Liberal E Spinu pi Liltz eet HE LES Defunct disk drive count Logical devices Failed Degraded SSDs assigned to MaxIQ Cache pool aximum SSDs allowed in MaxIQ Cache pool MaxIQ Read Cache Pool Size es DO cacas Estela rats MaxIQ Cache Read Write Balance Factor NCO status Statistics data collection mode Optimal SAS SATA Adaptec 5405 256 MB Disabled Disabled Enabled High Default Dynamic Disabled 0 0 0 2 0 0 0 8 0 000 GB 0 Srl Enabled Enabled Controller Version Information BIOS Firmware Driver Boot Flash EH LAZIZ 2461 18252 On
35. those new LUNs If this parameter is omitted it defaults to the global parameter auto add mapped luns 8 12 2 The delete Command delete lt wwn gt Delete the Node ACL with the specified wwn lt wwn gt must be a valid existing WWN 8 13 The Node ACLs Context 8 13 1 The create Command create mapped lun tpg lun iscsi write protect Create a mapping for an existing TPG LUN to be exported to the initiator referenced by the ACL mapped lun is the Mapped LUN that appears for the initiator tpg lun iscsi is the corresponding TPG LUN iSCSI only Copyright O 2015 Datera 41 LIO Admin Manual MATERA lt write_protect gt is an optional Boolean parameter that specifies whether the initiator will have write access to the Mapped LUN 8 13 2 The delete Command delete lt mapped_lun gt Delete the specified Mapped LUN from the TPG lt mapped_lun gt must be a integer matching an existing Mapped LUN 8 13 3 The auth Config Group CHAP authentication is enforced by default for each explicit initiator NodeACL that is created Enforcing CHAP authentication is controlled by the TPG authentication attribute and the minimum requirement for initiators is one way authentication Initiator NodeACL authentication also depends on having CHAP set in the AuthMethod TPG attribute list which is the default for each TPG Warning CHAP authentication can be disabled for all initiator NodeACLs on a TPG endpoint basis
36. 0 Storage Object G id G6 oooosocoocooocoocoocooooocoocooocoocoononcooooooooo 0 Storage Object OH ed UES rece cete coo 0 5 0 5 0 0 0 0 0 EE Ee CED D D 9 0 0 0 8 0 Storage Object aor NINE Ge cor EE EE 0 Target DATA rc c ava guage MP 0 Target EE 0 Target A EE 3L marger O 21 800 80 0522 93158 EE EE ccoo EELER enabled O SUE 000000600 0008000000000000000000050000000000 00 00000000000000 1 ACL ESSE O e E o a a le eM S SEES TES 1 Mapped LUN O mapa lun uogunmonadoodsboo000Uv0doo0050009900999999990555 lunO rw gr LUGS ssnscocobO000c0cocooucond u sbbpDO000909 U00O00nPD D BOOSH pDoocooooo99026596 1 LUN G lum isacocobobocoococonnuanobb bpcooncodsatna iblock my disk dev sdb ZS Copyright O 2015 Datera 17 LIO Admin Manual DATERA 5 4 InfiniBand SRP 5 4 1 Cookbook Command Comment ip srpt create lt WWPN gt Create an SRP target In ip_srpt lt WWPN gt Export the LUN my_disk luns create backstores iblock my_disk In ip_srpt lt WWPN gt Allow access for the initiator at lt WWPN gt acls create Initiator WWPN gt Table 4 Setting up an SRP target 5 4 2 Instantiate an SRP Target Alternatively instantiate an SRP target on the backstore device The SRP ports available on the storage array might be presented with the following WWPNs e 0x00000000000000000002c903000e8acd e 0x00000000000000000002c903000e8ace backstores iblock my disk gt ip_srpt create 0x00000000000000000002c903000e8acd Created t
37. 1 To disable Zero from the ESX 5 CLI esxcfg advcfg s 0 DataMover HardwareAcceleratedInit Value of HardwareAcceleratedInit is 0 This change takes immediate effect without requiring a Rescan All from VMware 11 3 3 Clone This is the signature VAAI command Instead of reading each block of data from the array and then writing it back the ESX hypervisor can command the array to duplicate a range of data on its behalf If Clone is supported and enabled VMware operations like VM cloning and VM vMotion can become very fast Speed ups of a factor of ten or more are achievable particularly on fast flash based backstores over slow network links such as 1 GbE Clone uses the T10 EXTENDED COPY command and defaults to a 4 MB block size Feature presence can be verified from the VMware ESX 5 CLI esxcfg advcfg g DataMover HardwareAcceleratedMove Value of HardwareAcceleratedMove is 1 Copyright O 2015 Datera 59 LIO Admin Manual MATERA To disable Clone from the ESX 5 CLI esxcfg advcfg s 0 DataMover HardwareAcceleratedMove Value of HardwareAcceleratedMove is 0 This change takes immediate effect without requiring a Rescan All from VMware 11 3 4 Delete VMFS operations like cloning and vMotion didn t include any hints to the storage array to clear unused VMFS space Hence some of the biggest storage operations couldn t be accelerated or thinned out Delete uses the T10 UNMAP comm
38. 10 The LUNs Context There are no command or parameter groups that apply to this context Copyright 2015 Datera 40 LIO Admin Manual NATERA 8 11 The luns Context 8 11 1 The create Command create storage object lun add mapped luns Create a new LUN in the TPG and export it as a new storage object storage object specifies the full path to the new storage object to be created Example For the storage object mydisk in the virtual HBA pscsiO the full path would be backstore pscsi0 mydisk lun is an optional parameter specifying the LUN number If omitted the first available LUN in the TPG is used lt add_mapped_luns gt is an optional Boolean parameter that specifies whether mapped LUNs should be created for all existing node ACLs thus automatically mapping those new LUNs If this parameter is omitted it defaults to the global parameter auto add mapped luns 8 11 2 The delete Command delete lun Delete the specified LUN from the TPG lun must be an integer matching an existing LUN 8 12 The Node acls Context 8 12 1 The create Command create lt wwn gt add mapped lun Create a Node ACL for the initiator node with the specified wwn lt wwn gt must match the expected WWN type of the target s fabric module add mapped luns is an optional Boolean parameter that specifies whether mapped LUNs should be created for all existing node ACLs thus automatically mapping
39. 19 5ooooocococooo MECA EE enabled Q ACLS oo ore EE o 0000 00 ODO ooo AAA Cc C 0 00 0 0 0 D 5 5 0 ACL EE 2 LUNs O M cr TE iblock block backend dev sdb G LW sooooccooococooccocoooccoco Filelio filelo lbackeme Jussu ii lei Aisesi icm 20 gt o gt 7 06 812 15 tipati Instantiate a second iSCSI target using another default IQN and add a backstore PSCSI to it iscsi iqn 20 70c82fb tpgt1 gt cd iscsi iscsi create Created target ign 2003 01 org linux iscsi san01 x8664 sn bf919196ff4e selected mes me La nue ec sk create Wee Lo iscsi gt iqn 2003 01 org linux iscsi san01 x8664 sn bf919196ff4e tpgt1 luns create backstores pscsi pscsi_backend Sele exc emm UNI Successfully created LUN 0 iscsi cd Taking you back to iscsi iqn 2003 01 org linux iscsi san01 x8664 sn 35ee770c82fb tpgtl Sea 16 DT 710 92 ETS tipati Targetcli per default automatically adds another TPG and per default assigns a sequentially increasing TPG tag starting again from 1 thereby adding another TPG 1 to the second ISCSI target For the new LUN targetcli also automatically assigns sequentially increasing LUN numbers starting from 0 thereby exporting LUN O in the example above The LUN ID can be set by using create with lun 0 9 4 SCSI Network Portals Assign an IPv4 address to the iSCSI TPG to form a valid iSCSI Endpoint iscsi iqn 20 70c82fb tpgtl portals create 192 168 62 151 Usine cereal WP port 3
40. 260 Successfully created network portal 192 168 62 151 3260 isesi iem 20 o 5 70682 1t5 tio 12 Targetcli automatically uses the SCSI default port number of 3260 thereby forming a valid new iSCSI Endpoint Display the resulting iSCSI targets iscsi iqn 20 70c82fb tpgt1 gt ls iscsi O DDR 2 Targets o ica 2005 01 org limu 18 es exeun 538664 85 3988711088219 5ooooocaaao IL TEE ORAR A a EE EE enabled EE 0 ACL O IMMUNE nouus oo ogaamdeddm abono UUOd0209004 09029511505 0 00 0 ae 2 LUNs eebe iblock block backend dev sdb Gr Jul ea aooo aoto o ea casas Tileio filelo backencl usse lei O portals aaa S RM PEE cas e S ola auna a pita ada AL Porra Copyright O 2015 Datera 46 LIO Admin Manual NATERA ec EE OK O acm 2 0105 0 1 oc limm lt 1SeSi eua 1L 00648 ena loi EL O3 I oooovnoroocoo LL WEE A EE EE EE enabled Om EE 0 ACL Oe EECH 1 LUN Faber pscsi pscsi backend dev sr0 o portals atea ano iss cost ne I MEE dL Portel Se OK isesi icm 20 7 0682 Ee Eeer 9 5 Access Control There are three basic setups for LUN authentication demo mode CHAP Initiator authentication and Mutual CHAP authentication Each are discussed below 9 5 1 Demo mode Demo mode describes a security configuration in which all access control to the LUNs in a TPG is disabled Open LUNs use automatic ACLs without authentication in their TPG context Such a configurat
41. 2c903000e8acd acls create 0x00000000000000000002c903000e8be9 Successfully created Node ACL for 0x00000000000000000002c903000e8be9 Created mapped LUN 0 Copyright 2015 Datera 18 LIO Admin Manual DATERA Entering new node ib srpt 0x00000000000000000002c903000e8acd acls 0x00000000000000000002c903000e8be9 ib_srpt 0x00 2c903000e8be9 gt ed gt The targetcli shell then automatically adds the appropriate mapped LUNs per default 5 4 5 Display the Object Hierarchy The resulting object hierarchy looks as follows displayed from the root object or E UU C EE G 19H IER 5500090 00000009090000000000400000000090 00 0 0000040000000 A ONS o D T H RT o a A A a E N a a A A S 0 Storage OoJect EE 1 Storage Object os my Gli socospoococococonaoooocooocoooo oooO oooO dev sdb activated Oi EE 0 Storage Object EE 0 Storage Object Gy i5gL MES aseos oo ooo ona ooo oo a o ooo oca o ona ao OOOO OOOH DOD 0 Storage Object WS EEN 1 Target o 0 990000900000901 90000002c599595000 899l soocosccoccceaceccoococsoccccaa enabled G Wels 552oo000000090000000000900000000029000000009090 090000522250009 1 ACL G Gear DCH Eller sss2soscoscocseoecoccc 1 Mapped LUN E lw conc gb op nacdooon gov edb n Does ooo OHO DODO Db OS 6 lunO rw LUGE 5530 0099 0 0 0 0 COO 0 0 0 d d 00000900090 0 0 O DD O 0 D D D D C O 0 O 0 EE 1 LUN G lum ziacoooobbooococcocooconcuaooboboooGoc no iblock my disk dev sdb or eniro
42. 39 22 PM x Evaluation Mode 47 days remaining Mere Fleischmann Adding SCSI Targets in the Storage Adapters View The SCSI Initiator dialog appears Go to the Static Discovery tab gt Add and enter the pertinent information of the SCSI Target then click OK gt Close Copyright O 2015 Datera 69 LIO Admin Manual DATERA ign 1998 01 com vmware localhost 34f4895a Target discovery methods Send Targets Static Target f Add Static Target Server Software Initiator Properties 8 E Ex iSCSI Server 192 168 1 17 Port mo SCSI Target Name ra linux iscsi mercury x8664 sn df14887e331d Parent E Authentication may need to be configured before a session can be established with the specified target CHAP Advanced Configure il d Close Help ja d Figure 9 vSphere 5 Discovering and connecting to iSCSI Targets Press OK to get back to the Storage Adapter view in the main vSphere Client window In the Details section on the bottom right click the new iSCSI device and Attach it 12 4 2 Configuring CHAP Authentication In the iSCSI Initiator Properties dialog the CHAP button optionally brings up the CHAP Credentials dialog which allows configuring CHAP Authentication for the iSCSI connection s T CHAP Credentials Es All iSCSI targets are authenticated using these credentials unless otherwise sp
43. 5 CLI esxcfg advcfg g VMFS3 HardwareAcceleratedLocking Value of HardwareAcceleratedLocking is 1 VMware actually uses ATS depending on the underlying filesystem type and history On VAAI Hardware New VMFS 5 Upgraded VMFS 5 VMFS 3 Single extent datastore ATS only ATS but fall back to SCSI 2 ATS but fall back to SCSI 2 reservations reservations reservations Multi extent datastore Only allow spanning on ATS except when locks on ATS except when locks on when locks on non head ATS hardware non head non head Table 7 Use of ATS depending on filesystem and history If anew VMFS 5 is created on a non ATS storage device SCSI 2 reservations will be used 2When creating a multi extent datastore where ATS is used the vCenter Server will filter out non ATS devices so that only devices that support the ATS primitive can be used 11 3 2 Zero Thin provisioning is difficult to get right because storage arrays don t know what s going on in the hosts VAAI includes a generic interface for communicating free space thus allowing large ranges of blocks to be zeroed out at once Zero uses the T10 WRITE SAME command and defaults to a 1 MB block size Zero only works for capacity inside a VMDK vSphere 5 can use WRITE SAME in conjunction with the T10 UNMAP command Feature presence can be verified from the VMware ESX 5 CLI esxcfg advcfg g DataMover HardwareAcceleratedInit Value of HardwareAcceleratedInit is
44. ALMeta ege iblock ib VSALarch dev vg_hdd lv_VSALarch o lun3 iblock ib VSALArchiveMeta dev vg_hdd lv_VSALArchiveMeta Gr PDOFEDILS cocoon oo ns oonnn doco oo sno OOOOH ODDO OOOO GOD OD ODO RO GOD 4 Portals SE OK Copyright O 2015 Datera 26 LIO Admin Manual MATERA NE SAN e OK SEET OK AA E SO EE OK acin 2003 01 cree JL stans 3 esa send 86643 Sm low T T ooocooos LL TEE O ene paaano noaa CO AO ROO COCO OO EHO OOO Co OUO EE enabled EEN 0 ACL O LUNS E 2 LUNs II re M ErETS iblock ib DBRedol dev vg ssd lv DBRedol I secun Less esr tenter eee Rcx iblock ib DBRedo2 dev vg ssd lv DBRedo2 ST SE lao ot der rer RCOo Sn HO Dco OO Cece RCA EE 0 Portai o isin 2003 01 owe Ltpes Ban El 8 Sm caca ssscocac CREE O HERG wil sosea ee scare ee s e aE eL eS Ss EE EE enabled EE 0 ACL EE 3 LUNs Se iie a5oococooococscos iblock ib DBBackup dev vg hdd lv DBBackup IE S iblock ib DBFlashl dev vg hdd lv DBFlashl Se ee EE iblock ib DBFlash2 dev vg_hdd lv_DBFlash2 O POL ALS ee EE Lal SOR SLS THE T H 0 589 ae sits OK O BIST EE EE EE EES OK EE OK EH IT OK ee 0 Target LA Peres E E EE 10 Target ZS 6 9 Best Practices 6 9 1 LUNs per TPG For 1 GbE iSCSI targets Datera recommends using no more than 4 8 LUNs per TPG For 10 GbE iSCSI targets Datera recommends using no more than 8 16 LUNs per TPG Each TPG uses a single Initiator_Target nexus for all lo
45. ATUS SUD LLE Delete Status Sois Copyright O 2015 Datera 60 LIO Admin Manual NATERA 11 4 Performance 11 4 1 Overview Performance improvements offered by VAAI can be grouped into three categories e Reduced time to complete VM cloning and Block Zeroing operations e Reduced use of server compute and storage network resources e Improved scalability of VMFS datastores in terms of the number of VMs per datastore and the number of ESX servers attached to a datastore The actual improvement seen in any given environment depends on a number of factors discussed in the following section In some environments improvement may be small 11 4 2 Cloning migrating and zeroing VMs The biggest factor for Full Copy and Block Zeroing operations is whether the limiting factor is on the front end or the back end of the storage controller If the throughput of the storage network is slower than the backstore can handle offloading the bulk work of reading and writing virtual disks for cloning and migration and writings zeroes for virtual disk initialization can help immensely One example where substantial improvement is likely is when the ESX servers use 1 GbE iSCSI to connect to a Linux storage system with SSDs or flash memory The front end at 1 Gbps doesn t support enough throughput to saturate the back end When cloning or zeroing is offloaded however only small commands with small payload go across the front while the actual I O is com
46. I start the iSCSI Initiator snap in and select the Discovery tab In the discovery window click Discover Portal and the Discover Target Portal dialog appears Enter the IP address or DNS name and Port number the pre filled iSCSI default is 3260 of your iSCSI target and click OK Targets Discovery Favorite Targets Volumes and Devices RADIUS Configuration Target portals The system will look for Targets on following portals Address Port Adapter IP address 10 100 100 3 3260 Default Default 192 168 1 29 3260 Default Default r To add a target portal dick Discover Portal Discover Portal LTE Portal mg mmm To remove a target portal select the address above and Remove Enter the IP address or DNS name and port number of the portal you then dick Remove want to add To change the default settings of the discovery of the target portal click sus servers the Advanced button i The system is registered on the following iSNS servers Refresh f Ma E IP address or DNS name Port Default is 3260 192 168 1 59 3260 MM CS f To add an SNS server dick Add Server Add Server To remove an iSNS server select the server above and TE then dick Remove More about Discovery and iSNS d X T 1 Figure 3 Windows 7 Discovering iSCSI targets Back in the discovery windo
47. Logical Disk Manager can access it Select disks Use the following partition style for the selected disks 9 MBR Master Boot Record l GPT GUID Partition Table Note The GPT partition style is not recognized by all previous versions of Windows It is recommended for disks larger than 2TB or disks used on Itanium based computers Lo Figure 7 Windows 7 Initializing a LUN After the disk is initialized by writing the MBR right click on the new volume select format and mark the partition as active For more technical information including iSCSI tuning parameters refer to the Windows iSCSI Initiator manual 9 Copyright O 2015 Datera 67 LIO Admin Manual NATERA 12 3 Microsoft Windows Server 2012 SRP Initiator First make sure that your Windows Server 2012 installation is current with all available updates Then follow these steps to install the Mellanox OFED 3 1 SRP Initiator for Windows Server 2012 Create the folder C Program Files OFED Copy ofed msi into the folder C Program Files OFED Install ofed msi from C Program Files OFED Disable the IPoIB adapter s Disable Mellanox Connect X VPI Reboot Windows Server 2012 into Safe Mode Update the following drivers o The Mellanox Connect X VPI driver Click on Update Browse My Computer to look for the new driver Pick the new driver from the list of available drivers
48. MATERA 3 6 2 LSI A drive failure condition can be identified as follows MegaCli PDList a lt controller gt egres Adapter Enclosure Slot Inquiry Firmware Error Failure Event After the failed drive has been replaced with a standby add a new dedicated hot spare drive MegaCli PDHSP Set Dedicated Array lt drive group gt PhysDrv lt enclosure slot al a lt controller gt For instance add a new hot spare in enclosure 0 drive 6 to drive group 0 on controller 0 MegaCli PDHSP Set Dedicated Array0 PhysDrv 0 6 a0 3 7 Best Practices 3 7 1 RAID Construction Considerations Datera recommends against configuring SSDs in RAID10 sets as that involves a significant loss of available net storage capacity which is rather expensive with SSDs and endurance without much performance gain In practice the performance gain IOPS of SSD RAID10 configurations is typically less than 15 at the expense of a loss of 50 of net storage capacity Instead Datera recommends using RAID5 with hot spare drives for SSD based volume groups The performance loss of RAIDS vs RAID10 in terms of latency and IOPS is typically less than 15 and considering the much higher performance levels of SSDs doesn t really matter in practice For SSD speed and endurance Datera recommends using a stripe size of either 64kB or 128kB depending on the SSD size in conjunction with RAIDS As a further optimization Dat
49. RAID5 set for enclosure 0 drives 0 2 with writeback read ahead direct cache and hotspare assigned on port 3 on installed adapter 0 i WMiSGaClla Cire DMA 1 5 020 091 092 Wi RA Direct helss3 80 3 4 Deleting a RAID Set Warning Deleting a RAID set cannot be reverted All data on the RAID set will be lost Only delete a RAID set if you are certain what you are doing 3 4 1 Adaptec Use GETCONFIG to get a listing of all logical drives available on a RAID controller arcconf GETCONFIG lt controller gt LD Permanently remove a logical drive from a RAID set on a RAID controller arcconf DELETE lt controller gt logicaldrive lt drive gt 3 4 2 LSI Use PDList to get a listing of all logical drive ports available on a RAID controller f MegaCli PDList aALL egrep Adapter Enclosure Slot Inquiry Permanently remove a logical drive from a RAID set on a RAID controller MegaCli CfgLdDel L lt drive gt a lt controller gt 3 5 Expanding a RAID Set 3 5 1 Adaptec List the logical drives that are available on a RAID controller arcconf GETCONFIG lt controller gt LD List the available disk drive slots and pair down the information a bit arcconf GETCONFIG 1 PD egrep Device State gt Reported Location INejoorinec Caas SM ART Wuesse Identify the slot numbers of the drive arcconf IDENTIFY lt controller gt DEVICE lt channel gt lt sl
50. Technical Documentation DAT E R A Administrator e Manual Version 4 0 Print Date 6 August 2015 Copyright O 2015 Datera Inc LIO Admin Manual MATERA Copyright Copyright 2015 Datera Inc All rights reserved No part of this publication may be reproduced stored in a retrieval system or transmitted in any form or by any means electronic mechanical photocopying recording or otherwise without the prior written consent of Datera Inc 2570 W El Camino Real Suite 380 Mountain View CA 94040 Trademarks Datera LIO and the Datera logo are trademarks of Datera Inc which may be registered in some jurisdictions Microsoft and Windows are trademarks of Microsoft Corp in the US and other countries used under license vSphere is a trademark of VMware Inc in the US and other countries used under license All other trademarks are the property of their respective owners Changes The material in this document is for information only and is subject to change without notice While reasonable efforts have been made in the preparation of this document to assure its accuracy Datera Inc assumes no liability resulting from errors or omissions in this document or from the use of the information contained herein Datera Inc reserves the right to make changes in the product design without reservation and without notification to its users Disclaimer If this product directs you to copy materials you must have permissi
51. The data on the resulting open LUN may be compromised unless other security means are in place such as physical security by deploying a separate SAN network Disable CHAP authentication only if you are sure what you are doing see also Section 9 5 Note CHAP authentication is disabled by default in TPG demo mode when it is configured with generate node acl 1 e mutual password lt string gt Default value The password for mutual iSCSI initiator NodeACL authentication e mutual_userid lt string gt Default value The username for mutual iSCSI initiator NodeACL authentication e password lt string gt Default value The password for one way iSCSI initiator NodeACL authentication e Userid lt string gt Default value The username for one way iSCSI initiator NodeACL authentication 8 13 4 The attribute Config Group The following attributes are specific to Initaitor NodeACL context e dataout timeout integer Values 0 60 default value 3 The timeout value in seconds for an outstanding Data Out request before invoking recovery e dataout timeout retries integer Values 0 15 default value 5 The number of data out timeout recovery attempts before invoking a failed path e default erl integer Values 0 2 default value O Copyright O 2015 Datera 42 LIO Admin Manual MATERA The default ErrorRecoveryLevel to enforce for this Initiator NodeACL
52. aMover MaxHWTRansferSize Value of MaxHWTransferSize is 4096 For vSphere 5 the default chunk size is 4096 kB Increasing the VAAI Clone chunk size causes more data to be copied with a single EXTENDED COPY command thereby reducing protocol overhead and the overall time required for the Clone operation esxcfg advcfg DataMover MaxHWTRansferSize s 16384 Value of MaxHWTransferSize is 16384 The best performance is usually achieved with the maximum chunk size of 16384 kB The LIO SCSI Target also allows adjusting the command queue size i e the maximum number of outstanding SCSI commands in flight For optimal performance the queue size should be set to the maximum value From the targetcli shell in the corresponding TPG context the procedure is as follows iscsi iqn 20 a0e4all tpgtl set attribute default cmdsn depth 128 Mises akrepato DT Enel ocios See also Section 9 5 Copyright O 2015 Datera 63 LIO Admin Manual DATERA 12 Initiator Setup 12 1 Overview This section describes how to setup Windows Linux and VirtualBox iSCSI initiators to login into Linux based SANs Instructions for Fibre Channel and InfiniBand initiators will be added in the future 12 2 Microsoft Windows 7 iSCSI Initiator 12 2 1 Connecting to Targets To connect to an iSCSI target using Microsoft Windows 7 or Windows Server 2008 R2 the built in iSCSI initiator can be used Go to Start gt Control Panel search for iSCS
53. ac 0 Target GS oop sack ee Met E ERE RURSUM NR 0 Target A oo quc mE 10 Target ZS 5 5 InfiniBand iSER 5 5 1 Cookbook Command Comment iscsi create Create an iSCSI target In iscsi lt IQN gt tpgt1 Associate an lt P_address gt portals create lt IP_address gt In iscsi lt IQN gt tpgt1 lt IP_address port gt Enable iSER iser_enable In iscsi lt IQN gt tpgt1 Export the LUN my_disk luns create backstores iblock my_disk In iscsi IQN tpgt1 Enable Demo Mode set attribute authentication 0 Beware demo mode write _protect 0 generate node acls 1 cache dynamic acls 1 Table 5 Setting up an iSER target Copyright O 2015 Datera 19 LIO Admin Manual MATERA 5 5 2 Instantiate an SER Target Instantiate an SER target on the backstore device to form a Target Portal Group TPG backstores iblock my disk iscsi create Created target iqn 2003 01 org linux iscsi targetcli demo x8664 sn 05135a0e4all selected WIE aer La nueces IN create Wee IL Entering new node iscsi igqn 2003 01 org linux iscsi targetcli demo x8664 sn 05135a0e4all tpgtl iscsi iqn 20 a0e4a11 tpgti1 Targetcli creates the TPG automatically assigns the next default TPG tag 1 and changes the working context to the resulting tagged TPG Assign an IP address here IPv4 192 168 1 139 to the TPG to make it accessible to iSER initiators iscsi iqn 20 a0e4all tpgtl portals create 192 168 1 139 sing
54. ag EUI Extended Unique Identifier A 64 bit number that uniquely identifies every device in the world The format consists of 24 bits that are unique to a given company and 40 bits assigned by the company to each device it builds Initiator The originating end of a SCSI session Typically a controlling device such as a computer IPS Internet Protocol Storage The class of protocols or devices that use the IP protocol to move data in a storage network FCIP iFCP and iSCSI are all examples of IPS protocols IQN iSCSI Qualified Name A name format for iSCSI that uniquely identifies every device in the world e g iqn 5886 com acme tapedrive sn a12345678 ISID Initiator Session Identifier A 48 bit number generated by the Initiator that uniquely identifies a session between the Initiator and the Target This value is created during the login process and is sent to the target with a Login PDU MC S Multiple Connections per Session A part of the iSCSI specification that allows multiple TCP IP connections between an Initiator and a Target MPIO MultiPah I O A method by which data can take multiple redundant paths between a server and storage Network Portal The combination of an iSCSI Endpoint with an IP address plus a TCP port The TCP port number for the iSCSI protocol defined by the IANA is 3260 OUI Organizationally Unique Identifier A 24 bit number that is purchased from the IEEE Registration Authority This identif
55. al CHAP authentication and one in Demo Mode look as follows isesi gt ls OS EET TET 2 Targets o igm 2003 01 org linuxz isesi sam0l 06643 em Dee 7 70682119 ocoroocsaVo LL WEE PO SEO ee sat aaa EE EELER enabled OH COS prurito oo ooo SU HD DODD a OOO OG GORD OOOO o0 5mbbpoDOO 52560096 1 ACL Qe em dL 99 05 s com malemos irics Wom t4lOs oo cooocoonscsooovooocoooc 2 Mapped LUNs massa lun 55 5056oo50nocoocobooocoosocoosooococooooooocoooco lunO rw O mapeo Il 5555550cacocn noccocobooooccococanocooooooo oncooo lunl rw EE EEUU 2 LUNs Copyright O 2015 Datera 49 LIO Admin Manual MATERA G RET H iblock block backend dev sdb o tuni ete eL aa Ted Leite Leben lackend usr sre iileirc O portals coos oiocodgo0B500dd0005000000n50025000050029290400009900 3L Portal Om OD 168 62 Ode rere ccc e e esee EE OK o auem 2009 01 026 M abiisse isesi exeun 5381604 8 em lox 9191961149 oooooocaaao LL TEE OR AChE Sateen ooo do DE RHA DOO DOO oO boohoo ooo ao AA enabled O ACTS EE 0 ACL OS IMIS aooboooooos Ee ORO EE OS TOO EET 1 LUN e pscsi pscsi backend dev sr0 OP ES atonsopoctobeosandonobponnoooonoo saga pomooosobenaad LI Portal el OK isEesi gt For configuring explicit Node ACL authentication see Section 8 13 3 For the corresponding Microsoft Windows iSCSI initiator configuration see Section 12 2 2 and for the corresponding VMware vSphere iSCSI initiato
56. amg iscsi fabrie mocule ZS 5 1 3 Display the Initial Object Hierarchy The initial object hierarchy is empty gt 1s O A O E a RETO T 9 INC EE EE Lo OS FETO oe eue ee eR ene rete teas eese oie a aaa sr ee HS 0 Storage Object Al TT 0 Storage OoJect E O EEN 0 Storage Object O zel d oooooocoocooooocoocooonoococooooooocoononcoooboooooo 0 Storage Object Os teclea oe tec o D D 9 0 0 0 0 0 0 EE EE 0 Storage Object EECHER 0 Target DS EE cy theese EAE Pare MAP EP CF e CUI E 0 Target EE EEN 0 Target O EE 10 Target ZS Copyright O 2015 Datera 13 LIO Admin Manual DATERA 5 1 4 Create a Backstore First create the underlying backstore device here my_disk on the physical SCSI disk device dev sdb gt backstores iblock create my disk dev sdb Generating a wwn serial Created iblock storage object my disk using dev sdb Entering new node backstores iblock my disk backstores iblock my disk Alternatively a backstore can also be created on an LVM volume gt backstores iblock create my disk dev vg0 lvl Generating a wwn serial Created iblock storage object my disk using dev vg0 lvl Entering new node backstores iblock my disk backstores iblock my disk In either case targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it 5 2 iSCSI 5 2 4 Cookbook Command Comment iscsi create Create an SCSI target In
57. and hardware drivers 2 2 Fabrics The LIO Linux SCSI Target supports the following network fabrics and protocols for exporting storage devices backstores 2 2 1 SCSI iSCSI Internet SCSI encapsulates the Small Computer System Interface SCSI protocol in TCP IP packets to link storage devices over commodity IP networks LIO can provide SANs over any standard IP network infrastructure including LANs WANs cellular and the Internet A Note Datera strongly recommends against using iSCSI TOE functionality with LIO 2 2 2 Fibre Channel Fibre Channel FC is a low latency high performance storage area network SAN protocol and fabric LIO supports Fibre Channel with the Host Bus Adapters HBAs from QLogic e QLogic 2400 Series single port QLE246x 4GBFC e QLogic 2500 Series dual port QLE256x 8GBFC e QLogic 2600 Series dual port 16 GBFC incl high speed and SR IOV support Copyright 2015 Datera 2 LIO Admin Manual MATERA 2 2 3 Fibre Channel over Ethernet Fibre Channel over Ethernet FCoE tunnels Fibre Channel packets traffic through enhanced Ethernet networks FCoE is layered directly on top of Ethernet i e it doesn t use IP and thus isn t routable As a result FCoE storage devices cannot be accessed beyond the Ethernet subnet that they reside on and they require specific FCoE enabled switches 2 2 4 InfiniBand SRP SRP is a high performance communication protocol that allows the transport of SCSI
58. and to allow thin capable arrays to offload clearing unused VMFS space However vCenter 5 doesn t correctly handle waiting for the storage array to return the UNMAP command status so the use of Delete is disabled per default in vSphere 5 Feature presence can be verified from the VMware ESX 5 CLI esxcli server my esx server u root system settings advanced list option NMFS3 EnableBlockDelete Type integer Int Value O0 Default Int Value O0 Min Value 0 Max Value 1 String Value Default String Value Valid Characters Description Enable VMES block delete To enable Delete from the ESX 5 CLI see also 16 esxcli server my esx server u root system settings advanced set int value 1 option VMFS3 EnableBlockDelete Value of EnableBlockDelete is 1 Many SATA SSDs have issues handling UNMAP properly so it is disabled per default in LIO To enable UNMAP start the targetcli shell enter the context of the corresponding backstore device and set the emulate tpu attribute backstores iblock fioa set attribute emulate tpu 1 Baramejterfenulla tee US mow Vil backstores iblock fioa Reboot the ESX host or re login into the backstore in order to get the Delete operation as recognized then verify its presence from the VMware ESX 5 CLI 4 esxcli storage core device vaai status get naa 6001405a2e547c17329487b865dla66e VAAI Plugin Name TS Status HI ie Clome Status SUO Aero ST
59. aqua Lena nh Read cida Rena cena vetas 11 4 4 Creating Logical Volum s n ete Aida tounge 11 4 5 Deleting Logical Vols s ereere neen e beer eoe te eo oet eese deeg s peu td 11 4 6 Resizing Logical Volumes oc ere rete dee degt Seege ess ie Ive ona 11 4 7 Resizing Physical Volumes cccssssccececessessaececececessesnaecececssessesaeaeeeeeceseesesaeaeeeeeessseseaeaeeeesens 12 4 8 Best Prati i e 12 5b Targetcli Quick Start Guide 1 1 0L e eerie ee Ee EE eo geeER epa pee eR Bana REESEN 13 SE ACAD O ON 13 PAM E 14 5 3 Fibre Channel dead deeg 16 5 4 InfiniBand SRP decere e cere eer oet teed veuve caia 18 5 5 aifibilBatid SER certae pee a eere e ee Pee Ue Een nn doa ers a od Lee egenum dez e re ze NY ETE EE 19 5 6 a l ege AE dau aa e ea ei ARENS Eege 21 A CT 99 Re AO 23 AN EE 23 6 2 Working With CONTEXTS cti dd dianas 23 6 3 Command Completion and Help 23 6 4 Gommand SynhtaXc anal 24 Copyright O 2015 Datera ii LIO Admin Manual MATERA 6 5 Command Le EE 24 6 6 Object Meana ARA ii 24 6 7 Creating a LUN and exporting iii o ai 25 6 8 Object Tree Elle cdta 25 6 9 Best Practica das 27 7 Targetcli COMMANGS wisccssscciccssssceccsssscecssssssessssssaeccsssssacsssstacccsssscecnssssaeessssesesnasssacsosssesnssasseanesssss 28 7 1 The bookmarks Commande 28 rote el ME 28 7 3 The exit Command EE 29 7 4 mege COMMANG nat cid 29 7 5 MEP COMMING TTT 29 TO TAS COMIMANG T 29 7 7 Thepwd Ee
60. arget 0x00000000000000000002c903000e8acd Entering new node ib srpt 0x00000000000000000002c903000e8acd ib srpt 0x00 2c903000e8acd Note that targetcli automatically changes the working context to the resulting tagged Endpoint 5 4 3 Export LUNs via Fibre Channel Next declare a LUN for the backstore device to form a valid SAN storage object ib srpt 0x00 2c903000e8acd luns create backstores iblock my disk Seleccecl IN 0 Successfully created LUN 0 Entering new node ib srpt 0x00000000000000000002c903000e8acd luns lunO ib srpt 0x00 acd luns lun0 Targetcli automatically assigns the default ID 0 to the LUN and then changes the working context to the new SAN storage object Now the InfiniBand target is created and exports dev sdb as LUN O Return to the underlying Endpoint as no attributes need to be set or modified for standard LUNs ib srpt 0x00 act luns lun0 ed Taking you back to ib srpt 0x00000000000000000002c903000e8acd ib srpt 0x00 2c903000e8acd 5 4 4 Define Access Rights Configure the access rights This involves setting up individual access rights for each initiator based in its WWPN First determine the WWPN for the respective SRP initiator For instance for Linux initiator systems use cat sys class infiniband ports gids 0 sed e s fe80 0x0000 e s g For a simple setup simply grant access to the initiator with the WWPN determined above ib srpt 0x00
61. dress and ip port lt ip_address gt is an optional parameter that specifies the IP address of the new iSCSI Network Portal If it is omitted the first IP address found matching the local hostname will be used lt ip_port gt is an optional parameter that specifies the IP port number of the new iSCSI Network Portal If it is omitted the default port for the target fabric will be used 8 16 2 The delete Command delete lt ip_address gt lt ip_port gt Delete the iSCSI Network Portal with the ip address and ip port Copyright 2015 Datera 43 LIO Admin Manual NATERA 9 Targetcli Examples This section includes a number of examples of how setup the target create objects such as backstores Endpoints and Target Portal Groups It also illustrates some syntax sugar that targetcli uses such as object directory history command history TAB completion and bookmarks Note These examples assume auto_cd_after_create false to better demonstrate how object paths i e execution contexts function including the different context and command use scenarios The default of auto_cd_after_create true automatically changes to the new object context upon creation of the associated object Also these examples are based on iSCSI but the concepts apply symmetrically to all other fabrics and protocols 9 1 Startup Targetcli can be invoked by running targetcli as root from the underlying Linux system shell targetcli Welcom
62. e to me targeteli CLs Copyright c 2014 by Datera Inc All riclicts reserved Visit us at http www datera io Using loopback fabric module Using allo SOE feric mecht Using igesi kalorie mocule Using qla2xxx fabric module AS Upon targetcli initialization the underlying RTSlib loads the installed fabric modules and creates the corresponding configfs mount points at sys kernel config target lt fabric gt as specified by the associated spec files located in var target fabric fabric spec Set auto_cd_after_create false to prevent targetcli from automatically changing the object context or working directory to new objects after their creation gt set global auto cd after create false Baremeter auto cd etter create de now false ZS This is the default behavior assumed for the following examples 9 2 Backstores Enter the top level backstore object and create a few backstores storage objects using IBLOCK FILEIO and PSCSI type devices gt cd backstores backstores gt Create an IBLOCK backstore from a dev sdb block device Note that this device can be any TYPE_DISK block device and can also use dev disk by id symlinks backstores gt iblock create dev dev sdb name block backend Generating a wwn serial Copyright O 2015 Datera 44 LIO Admin Manual MATERA Created iblock storage object block backend using dev sdb backstores Alternatively logical volume created by LVM ca
63. ecified in the target s CHAP settings Li The CHAP secret and Mutual CHAP secret must be different CHAP target authenticates host Select option z Mutual CHAP host authenticates target Select option Do not use CHAP y Ca oe e ll d Figure 10 vSphere 5 Configuring SCSI CHAP Authentication If physical security has been established for the SCSI network there is no need to enable CHAP as the VMware VMFS ensures data integrity for VMware data stores Consequently for VMware environments the LIO Linux SCSI Target should be configured in Demo Mode as described in Section 9 5 1 Copyright O 2015 Datera 70 LIO Admin Manual DATERA 12 4 3 Configuring LUNs Back in the Storage Adapters view of the main vSphere Client window select Storage to create VMFS data stores on the SCSI LUNs If the new LUNs don t yet show up klick Rescan All File Edit View Inventory Administration Plug ins Help D E ie Home b gf Inventory p m Hosts and Clusters ge geg ee B es El Ed WIN KF99MLLVB9Q El Ey VM Workbench El A 192 168 1 28 E Win SsD 192 168 1 30 Name Y RescanVMFS Y RescanVMFS Y Remove datastore Z Tasks Alarms Target H 192 168 130 H 192 168 130 H 192 168 1 28 192 168 1 28 VMware ESXi 5 0 0 469512 Evaluation 53 days remaining AO Te E TARTA a Configuration WES 211 a E E ES TT AL T
64. ected Data Transfer Length Received Immediate Data Length R2T is required as a default action unless both initiator and target send this key pair attribute specifying InitialR2T No Only the first outgoing data burst immediate data and or separate PDUs can be sent unsolicited i e not requiring an explicit R2T e MaxBurstLength integer Values 512 2 24 1 default value 262144 Copyright O 2015 Datera 39 LIO Admin Manual NATERA 8 9 5 The Initiator and target negotiate maximum SCSI data payload in bytes in a Data In or a solicited Data Out SCSI sequence A sequence consists of one or more consecutive Data In or Data Out PDUs that end with a Data In or Data Out PDU with the F bit set to 1 MaxConnections integer Values 1 65535 default value 1 This value controls the usage of MC S Initiator and target negotiate the maximum number of connections requested and or acceptable MaxOutstandingR2T integer Values 1 65535 default value 1 The Initiator and target negotiate the maximum number of outstanding R2Ts per task excluding any implied initial R2T that might be part of that task An R2T is considered outstanding until the last data PDU with the F bit set to 1 is transferred or a sequence reception timeout is encountered for that data sequence MaxRecvDataSegmentLength lt integer gt Values 512 2 24 1 default value 8192 typically jumps to gt 64535 for life session
65. ection 9 5 3 for the corresponding settings in the targetcli auth group e Enter the target secret matching the targetcli userid e optionally adjust the name matching the targetcli password Copyright O 2015 Datera 65 LIO Admin Manual DATERA n 5 wee A Advanced Settings General IPsec Connect using Local adapter Default Initiator IP Default Target portal IP Default CRC Checksum _ Data digest 1 Header digest V Enable CHAP log on CHAP Log on information CHAP helps ensure connection security by providing authentication between a target and an initiator To use specify the same name and CHAP secret that was configured on the target for this initiator The name will default to the Initiator Name of the system unless another name is specified Name ign 1991 05 com microsoft ibm t410s Target secret soccccccccccee Perform mutual authentication To use mutual CHAP either specify an initiator secret on the Configuration page or use RADIUS Use RADIUS to generate user authentication credentials F Use RADIUS to authenticate target credentials oe JL coma Figure 5 Windows 7 Configuring CHAP Authentication Then click the OK buttons to complete the connection to the target The LIO iSCSI Target will now appear as a Windows volume and if configured reconnect after reboots For more information on CHAP Authentica
66. ee 0 Target MO O EE EE EE 10 Target ZS 5 3 Fibre Channel 5 3 1 Cookbook Command Comment qla2xxx create lt WWPN gt Create a Fibre Channel target In gla2xxx WWPN Export the LUN my disk luns create backstores iblock my disk In gla2xxx WWPN Allow access for the initiator at lt WWPN gt acls create lt Initiator WWPN gt Table 3 Setting up a Fibre Channel target 5 3 2 Instantiate a Fibre Channel Target Alternatively instantiate a Fibre Channel target on the backstore device The Fibre Channel ports available on the storage array might be presented with the following WWPNs e 21 00 00 24 ff 31 4c 48 e 21 00 00 24 ff 31 4c 49 backstores iblock my disk qla2xxx create 21 00 00 24 ff 31 4c 48 Created target 21 00 00 24 ff 31 4c 48 Entering new node qla2xxx 21 00 00 24 ff 31 4c 48 ads Set Ae ES Sil les aes Note that targetcli automatically changes the working context to the resulting tagged Endpoint 5 3 3 Export LUNs via Fibre Channel Declare a LUN for the backstore device to form a valid SAN storage object qla2xxx 21 0 4 ff 31 4c 48 luns create backstores iblock my disk Sere cec eI UNIS Successfully created LUN 0 Entering new node qla2xxx 21 00 00 24 ff 31 4c 48 luns lun0 qla2xxx 21 0 48 1uns lun0 Targetcli automatically assigns the default ID 0 to the LUN and then changes the working context to the new SAN storage object Now the Fibre Channel target is crea
67. ees 46 9 5 ACCESS e ln EE 47 9 6 Discovery Control iia ege deen ege Seen 50 9 7 Object O E EE ENE 51 A O 52 9 9 Navigation and Auto Completion eee eee 52 9 10 Bookmarks cian TUILOLOO I DOLI 53 RTSlib Storage Management Library and API ssscccsssssceccssssceccssssceccesssssecesssssscessssesansees 55 y Lo Ps e EE 55 10 2 Fibre Clhiannel dieses ti pcd orte oc tr eoo ore a RA AAA A AAA 56 le eve I Rd 57 VMware VAMAI 4 eee eeee eese n ERR RR RE RE Epas RER EE RER sh EE SPESEN RRE hastas sess sess sese PSR NENESE ESES 58 TIT OVEViIEW E 58 T12 e 58 REG A gll Let 58 11A Le Gas co ec rte terio EDE RE ure UE ERE OI EOE OOA OOO ee TUR ORE FIRE 61 WD eS Stati SUES m 62 11 6 Beet PRACTICES iria ir A A AA 63 ugeet E 64 T21 e EVI 64 12 2 Microsoft Windows 7 iSCSI Initiator 64 12 3 Microsoft Windows Server 2012 SRP Initiator s sss eee eee eee 68 12 4 VMware vSphere 5 iSCSI Initiator 69 R SEU iSCSI WEL EE 72 12 6 VirtualBox iSCSI ln TE 73 GIOSSANY TT 74 References ie sssiscsesssccccssssiccessssccesssssiacsesssicessasiiacsssssscnsssssiacsesssienssassiacessdeesnssasstacsessssanssastiaeessses 76 Copyright 2015 Datera iv LIO Admin Manual MATERA 1 Introduction The target audience of this manual are system and storage administrators The intent of this manual is to provide the necessary backg
68. era recommends keeping the SSDs net storage capacity below the 8096 utilization mark which results in in more robust long term performance especially in comparison with RAID10 with less disk space For HDD arrays with an emphasis on high performance and fault tolerance RAID10 typically provides the best tradeoffs between performance reliability and price in a single shelf environment Depending on the block size of filesystem Datera recommends using a stripe size that equals 8x to 16x of the defined block size Example For a block size of 4KiB with an ext4 filesystem Datera recommends a stripe size of 32kB to 64kB as guidance If the filesystem typically uses larger files larger stripe sizes will result in higher sustained performance This is because the block pre allocation on the drive can be done in larger chunks and therefore reduce the amount of head seeks operations necessary to be performed on the HDDs Copyright O 2015 Datera 10 LIO Admin Manual MATERA 4 LVM Setup 4 1 Overview For ease of configuration and modularity while maintaining scalability Datera recommends using the Logical Volume Manager LVM 4 2 Initializing Disks or Disk Partitions Before using a drive or a drive partition as a physical volume it must be properly initialized pvcreate lt device gt Warning Not initializing the LVM device with pvcreate can cause subtle integrity errors on STOP the backstores that later result in system i
69. from the current path or in the current context if path is omitted parameters depend on the command and its context sensitive options 6 5 Command Chains Targetcli has the ability to chain commands which provides powerful semantics for creating complex command sequences Command chains are constructed by chaining multiple single commands together separated by a comma lt command1 gt lt command2 gt lt command3 gt When a command results in a context change i e enter target iqn 1999 03 org foobar 1234 enters that target context the next command in the chain executes in that new context In addition create context and enter context can be used in command chains enabling easy scripting of creation deletion and query operations of objects embedded deeply in the Target object tree 6 6 Object Tree The tree of objects managed by targetcli changes according to which objects are created or deleted and which fabric modules are loaded on the system Notably not all fabric modules have all possible features and functions A key difference of the iSCSI fabric module is that it has an intermediate level Target Portal Groups TPG between the fabric target defined by its IQN name and the ACL LUN and Network Portal objects This intermediate level does not exist for other fabric modules whose target node defined by its WWN in the format supported by the specific fabric module is directly attached to the lower objects
70. gins and thus shares the same processor core resources among all associated LUNs so too many LUNs in a single TPG can limit the overall performance per initiator 6 9 2 Command Queue Depth The fabric command queue depth is different from the TCQ depth of the storage object The fabric command queue depth is enforced by the target to determine how many outstanding I Os a given initiator can send to all of the LUNs for a given initiator on a given target Endpoint Once an I O has been cleared by the fabric command queue it still is enforced by the underlying storage object TCQ which is shared for the storage objects across all of the fabric module exports For larger storage arrays Datera recommends increasing the command queue depth to 128 To increase the command queue depth enter in the TPG context d Leed Zeng 2 0 5 ata pots set attribute Cereulrt emelsia cepta 128 6 9 3 Write Cache Emulation For iSCSI initiators that are not fully standards conformant such as with Red Hat RHEL5 emulation of the write cache is recommended on the IBLOCK device level To enable write cache emulation enter in the IBLOCK device context backstores iblock gt set emulate write cache 1 Copyright O 2015 Datera 27 LIO Admin Manual MATERA 7 Targetcli Commands 7 1 The bookmarks Command bookmarks lt action gt lt bookmark gt e Manage personal bookmarks e Bookmarks can also be accessed with the cd command For instance
71. h With only a group specified list all available parameters within that group With no group or parameters specified list all available groups Example set equlelsadll color mocde t ue loglevel ero me eI SEINS 7 10 The status Command status Display the current node s status summary 7 11 Variable Types The following variable types are used for targetcli command parameters and attributes bool Boolean Values TRUE FALSE bool iSCSI Boolean per RFC 3720 Values Yes No enum List of alphanumeric values integer Digits Values 0 9 string Alphanumeric characters color black blue cyan default green magenta red white yellow loglevel critical debug error l nfo warning 7 12 The global Config Group auto add mapped luns bool Default value TRUE If TRUE automatically creates node ACLs mapped LUNs after creating a new target LUN or a new node ACL auto cd after create bool Default value TRUE If TRUE changes current path to the new object that was just created auto enable tpgt bool Default value TRUE If TRUE automatically enables TPGTs upon creation color command color Default value cyan Color for command completions color default color Default value none Default color for text display The default is to use the terminal foreground color This is the preferred setting as individual user background colors can t be known
72. iSCSI Network Portals provide the access points to the Target An iSCSI Network Portal always requires an IP address to be accessible which needs to be available on the storage array before being associated with portal Create ACLs ACLs need to be defined for initiators to connect unless authentication is disabled via the corresponding attributes Disabling authentication is only available for iSCSI target and it is deprecated for production setups Set a LUN mapping ACLs allow the mapping of LUN IDs onto Mapped_LUN IDs which represent the LUN IDs presented to initiators These mappings implement preferred LUN IDs for particular initiators For instance LUN 1 can be mapped onto Mapped_LUN O to make LUN 1 appear as SCSI LUN O on the initiator Usually LUNs are identically mapped however 6 8 Object Tree Example targetcli Welcome to tae targerel a CIbg Copy ierg e 2004 low AD ci line ALL zigare reserved Visit Using Using Using Using o us at http www datera io loopback fabric module alo SPC algo mot Igesi Talore moule qla2xxx fabric module EE e all e EE EE 9 Simexece QOleyect l Copyright O 2015 Datera 25 LIO Admin Manual DATERA ors MAMA RA O O ie EC EE 18 Storage Objects Ho WEES Cc cq ooo dev vg hdd 1v DBBackup activated SE dev vg_ssd lv_DBData01 activated Om EE c e e e e MEN La E e dev vg_ssd lv_DBData02 activated O d
73. ier uniquely identifies a vendor referred to by the IEEE as the assignee globally and effectively reserves a block of each possible type of derivative identifier such as MAC addresses SAM SCSI Architectural Model A document that describes the behavior of SCSI in general terms allowing for different types of devices communicating over various media Target The receiving end of a SCSI session typically a device such as a disk drive tape drive or scanner Target Group A collection of fabric module SCSI target endpoints containing SCSI target ports that provide access to individual storage objects Target Port The combination of an iSCSI Endpoint with one or more LUNs TPG Target Portal Group A list of IP addresses and TCP port numbers that determines which interfaces a specific iSCSI target will listen to Copyright 2015 Datera 74 LIO Admin Manual NATERA TSID Target Session Identifier A 16 bit number generated by the target that uniquely identifies a session between the initiator and the target This value is created during the login process and is sent to the initiator with a Login Response PDU WWN World Wide Name A unique identifier that identifies a particular Fibre Channel or InfiniBand target Each WWN is an 8 byte number derived from an IEEE OUI and vendor supplied information Copyright O 2015 Datera 75 LIO Admin Manual NATERA 14 References 1 2 3 4 5
74. in advance color keyword color Default value cyan Copyright O 2015 Datera 30 LIO Admin Manual NATERA Color for keyword completions e color mode bool Default value TRUE Console color display mode Display in color if TRUE and display in black and white if FALSE e color parameter color Default value magenta Color for parameter completions e color path color Default value magenta Color for path completions e completions in columns bool Default value TRUE If TRUE display command and parameter completions in columns not lines e legacy hba view bool Default value TRUE If TRUE use the legacy HBA view which allows creating more than one storage object per HBA e logfile string Default value targetcli log tst Logfile to use Targetcli keeps its configuration and logs etc in the home directories under targetcli A dedicated user log file contains only session information for that user s sessions Users can share a log file and the corresponding session data will be merged e loglevel console loglevel Default value info Log level for messages going to the console e loglevel file loglevel Default value debug Log level for messages going to the log file e prompt length integer Default value 30 Maximum length of the shell prompt path O for infinite e tree max depth integer Default value O Maxi
75. ion is called demo mode because it usually only makes sense for simple demo setups Enable demo mode TPG operation for the PSCSI Endpoint iscsi iqn 20 70c82fb tpgt1 gt iscsi iqn 2003 01 org linux iscsi san01 x8664 sn bf919196ff4e tgptl set attribute authentication 0 demo mode write protect 0 generate node acls 1 cache dynamic acls 1 Parameter demo mode write protect is now 0 Parameter authentication is now 0 PALEMStGr GSmeracSs mocks acls 18 Mr v MNT Buggi chymemie acils 18 mow 11 MAisesi icm 20 s 7006812 15 tipati This exports the PSCSI backstore as LUN O without any access restrictions security and data integrity hazards Don t do this for production setups unless you are sure STOP Warning Exporting open LUNs with no authentication requirements creates significant what you are doing Datera strongly recommends using demo mode only under the following conditions e You have established physical security through a closed controlled SAN environment e You are using your SAN in conjunction with a clustered filesystem that guarantees coherence across multiple initiators typically via distributed locking e You have carefully analyzed your ACL setup with regard to its security and data integrity requirements and risks For access control configurations that are suitable for production setups please see below 9 5 2 CHAP Initiator Authentication Enable secure sessions for the i
76. ject O i50L MES nooo90000000005009096500r009090000004 000000995 0 Storage Object Q ib SPPE oonbosobocooooooooococononoocooocooooo nonopoobooootoocooDg5 0 Target EE 1 Target tenes ORTOS cles Se E HESE L 0048 emm Sbagc4 o dl mes OF e 612 EE enabled EEN 0 ACL SKA mee ENEE EE EE EE EE 1 LUN So mal nee o ooo iblock my disk dev sdb OS POBE LS seee aca I Ecc EE dL Portal Om EE OK iser enabled NN NN to EE 0 Target O EE 10 Target ZS 5 6 Persistence The target configuration without the user data contained on the LUNs can be persisted across reboots by invoking saveconfig from the root context gt saveconfig WARNING Seving targeteli cemo current em RARO NON Cisk wLLL Evite your boot settings Tae Current target COmirigmwueenti gu willl become ie Zeiten boot contigo Are you sure os yes yes Making backup of LIO gt Target ContigEs with timestamp 2011 11 24 REES Generated LIO Target contigs ete terget beckup lio backup 2011 11 AMS OS SIS Ss Making backup of Target_Core Mod ConfigFS with timestamp 2011 11 24 162372 10 88 719593 Generated Target Core Mad contigs ete terget beackip ten lacio 20 11 111 11 24 1689781 99 7995 5A Suscsss fully usdelted ceramic contig ete target lio startsein Success fully upate ceramic contig ete terget tem Sa sia ZS Copyright O 2015 Datera 21 LIO Admin Manual MATERA Note Without saveconfig the target configuration will be lost upon rebooting o
77. ker lt bool iSCSI gt Default value No Used to control the usage or the target to initiator markers on the connection e ImmediateData bool iSCSI Default value No typically goes to Yes for life sessions The Initiator and target negotiate support immediate data To turn immediate data off the initiator or target must state its desire to do so ImmediateData can be enabled if both the initiator and target set ImmediateData Yes If both ImmediateData and InitialR2T are set to Yes the default then only immediate data is accepted in the first burst If ImmediateData is set Yes and InitialR2T is set to No then the initiator may send unsolicited immediate data and or one unsolicited burst of Data Out PDUs If ImmediateData is set to No and nitialR2T is set to Yes then the initiator must not send unsolicited data and the target rejects unsolicited data with the corresponding response code If both mmediateData and InitialR2T are set to No then the initiator must not send unsolicited immediate data but may send one unsolicited burst of Data Out PDUs e InitialR2T bool iSCSI Default value Yes Used to turn off the default use of R2T for unidirectional and the output part of bidirectional commands thus allowing an initiator to start sending data to a target as if it has received an initial R2T with Buffer Offset Immediate Data Length and Desired Data Transfer Length MIN FirstBurstLength Exp
78. led as it is only used by initiators to query the available target endpoints If discovery authentication is enabled the initiator must provide one way authentication by sending matching password and userid login credentials If mutual_userid and mutual_password is also set the target must use mutual authentication by responding with its login credentials e enable lt bool gt Values 0 1 default value O disabled Controls enforcing iSCSI discovery authentication e mutual password lt string gt Default value The password for mutual iSCSI discovery authentication e mutual_userid lt string gt Default value The user name for mutual iSCSI discovery authentication Copyright 2015 Datera 35 LIO Admin Manual MATERA e password lt string gt Default value The password for one way iSCSI discovery authentication e userid lt string gt Default value The user name for one way iSCSI discovery authentication 8 8 The Target Context The Target context and contains all objects that belong to a specific fabric target iSCSI targets contain associated Target Portal Groups TPGs contexts identified by their TPG tags Thus the tpg tag iscsi parameter only applies to iSCSI targets For all other targets or fabrics the targetcli shell masks the TPG contexts LUNs are then exported for all fabrics via fabric specific Endpoints from their respective LUN object context
79. lthy EISA Configuration Disk Management P gt Task Scheduler Es Simple Basic Healthy Primary Partition L A k More Actions Event Viewer E Simple Basic Formatting il Shared Folders Es SCSI Disk E Simple Basic NTFS Healthy Primary Partition E Reliability and Performe 9 iSCSI Disk G Simple Basic NTFS Healthy Primary Partition ii Device Manager Es SQ004869V01 C Simple Basic NTFS Healthy System Boot Page File Active Crash Dump Pr H m D Disk 0 AAA AAA Basic SQ004869V01 C 298 09 GB 11 46 GB 288 90 GB NTFS 7 72 68 Online Healthy EISA Confi Healthy System Boot Page File Ac Healthy Primary Partitio Disk 1 AA ASS Basic iSCSI Disk G 953 MB 951 MB NTFS Online Healthy Primary Partition Disk 2 Basic 74 53 GB 74 53 GB Online Formatting 4 amp Storage te Disk Management Z Services and Applications m iSCSI Disk E 194 MB NTFS Healthy Primary Partition Disk 4 AAA Removable 490 MB 489 MB Online 2cp Romo lll Unallocated BW Primary partition Figure 6 Windows 7 Configuring LUNs such as iSCSI LUNs Disk 1 Disk 2 Disk 3 The first time Disk Management starts with a new connection and the disk area has not been previously configured the Disk Manager will prompt to initialize the disk with a particular partition style Eessen K E M You must initialize a disk before
80. me needs to be resized it is recommended to logout any attached initiators from this device as an initiator relogin is necessary to provide the new block device specifications lvresize L 1G lt vg_name gt lt lv_name gt Copyright 2015 Datera 11 LIO Admin Manual NATERA block device needs to be resized as well which is a filesystem dependent operation Warning Resizing the volume does not resize the drive contents so the file system on the TOF Resizing a volume with logged in initiators practically guarantees file system corruption Please refer to the system manuals on how to properly resize the file system on the block devices It is highly recommended to resize the filesystem via the iSCSI provided target and not on the storage node itself 4 7 Resizing Physical Volumes In case of extending the underlying RAID volume the size needs to be made available to the physical volume as well Using the command pvresize device expands the physical volume to the maximum value possible disk boundaries After expanding the physical volume the maximum value the new space will automatically be available to the corresponding volume group Note Datera recommends using LVM block devices to provide the backstores however the remainder of this manual uses examples with simple block devices 4 8 Best Practices Usually disabling the LVM write cache is a good idea Otherwise small problems with stale informati
81. moved in mainline v4 1 target code e default cmdsn depth integer Values 1 2 32 1 default value 16 Default iSCSI CmdSN session wide queue depth used by explict NodeACLs and dynamic NodeACLs with write protect 1 The value 16 is recommended for 1 GbE networks and the value 64 is recommended for 10 GbE networks e demo mode write protect bool Values 0 1 default value 1 enabled For initiator NodeACLs dynamically created with generate node acl 1 this attribute controls if LUNs appear with write protect 1 i e per default initiators only obtain read access to LUNs Warning Disabling demo mode write protection for a LUN exports that LUN with global read write access Concurrent writes from multiple initiators on that LUN can cause data corruption on it unless the writes accesses are serialized by a higher level protocol e g by a cluster file system such as VMware VMFS Disable this attribute only if you are sure what you are doing see also Section 9 5 e generate node acls bool Values 0 1 default value O disabled Allow all iSCSI initiators to login by dynamically generating an initiator NodeACL at login time Warning Enabling dynamic NodeACL creation for a LUN exposes that LUN to unauthenticated logins from initiators The resulting security hole for that LUN may compromise the data on it unless other security means are in place such as physical security by deploying a separate SAN
82. mum depth of displayed node tree e tree round nodes bool Default value TRUE Tree node display style e tree show root bool Default value TRUE Whether or not to display the root of the tree e tree status mode bool Default value TRUE Whether or not to display status in the tree Copyright O 2015 Datera 31 LIO Admin Manual MATERA 8 Targetcli Contexts 8 1 The Root Context 8 1 1 The saveconfig Command saveconfig Save the whole configuration tree to disk so that it will be restored on next boot Unless this command is executed changes are lost across reboots 8 1 2 The version Command version Display the targetcli version and the support libraries versions 8 2 The backstores Context There are no command or parameter groups that apply to this context 8 3 TheFILEIO Backstore Context 8 3 1 The create Command create lt name gt file or device lt size gt lt generate_wwn gt lt buffered gt Create a FILEIO storage object lt name gt is the name of the FILEIO new storage object lt file_or_device gt specifies the path to the file or a block device to be used for the storage backend lt size gt is an optional parameter that specifies the size of the file to be used if file_or_device depicts a regular file Otherwise if file_or_device depicts a block device size must be omitted The size syntax is e Ifsize is an integer it represents a number of bytes
83. n e cd navigates one step forward in the path history 7 3 The exit Command exit Exit the targetcli command line interface 7 4 The get Command get lt group gt lt parameter gt e List the value of one or more configuration parameter in the given group e With only a group specified list all available parameters within that group e With no group or parameter specified list all available groups Example Gert qulelesdl color moce loglevel comsole 7 5 help Command help lt topic gt Display the manual page for a topic or provides the list of available topics 7 6 The ls Command Is lt path gt lt depth gt Display either the nodes tree relative to path or relative to the current node lt path gt depicts the root path of the nodes tree to display path can be an absolute path a relative path or a bookmark lt depth gt specifies the maximum depth of the tree to display The default is 0 which displays the full complete tree 7 7 The pwd Command pwd Display the current path 7 8 The refresh Command refresh Refresh and update the objects tree from the current path 7 9 The set Command set lt group gt lt parameter gt lt value gt Copyright 2015 Datera 29 LIO Admin Manual NATERA Set one or more configuration parameters in the given group The global group contains all global CLI preferences Other groups are context sensitive i e depend on the current pat
84. n be used as a backstore For instance create an IBLOCK backstore from an LVM logical volume at dev volume group logical volume backstores gt iblock create dev dev vg0 lvl name block backend Generating a wwn serial Created iblock storage object block backend using dev vg0 lvl backstores gt Create an unbuffered FILEIO backstore with 2 GB size for the underlying file backstores gt fileio create name file backend file or dev usr src fileio size 2G Generating a wwn serial Not using buffered mode Created filelo file lnacisamel backstores gt Create a PSCSI SCSI pass through backstore for a physical SCSI device in this case a TYPE_ROM device using dev srO backstores pscsi create name pscsi backend dev dev sr0 Created SSSI storage object pacsi lone using ekew sic backstores gt Display the resulting active backstores backstores gt ls OS IEE ESE NS BER OS PAN EE 1 Storage Object o ile backend ooooocooooooconcoooooocooooooco usr sre tileio cdesctivatecl ee 1 Storage Object o block lsacizamd ocooocooccooccoaacoocoooccocccocaann20056 dev sdb deactivated G A eee uiv S e suse ene em etm Ses reete NR IER EIUS 1 Storage Object OF psesi_lacikend coooocooooncoooooocooocoonooocooooooco dev sr0 deactivated Ge 359b QUA sopor E EE d o HOC HO O70 0 0 0 0 0 0 0 0 0 coo 0 6 079 eo 0 Storage Object O rE MEP ooocoocscoococoooooooooocn onoooooooooc oognonopooooo
85. n testing to 128 because the maximum addressable LUN size in ESX is 2 TB which means that each VM can occupy a maximum of 16 GB including virtual disk virtual swap and any other files Virtual disks much smaller than this generally do not allow enough space to be practical for an OS and any application Load was generated and measured on the VMs by using iometer For some tests all VMs had load In others such as when sets of VMs were started stopped or suspended load was placed only on VMs that stayed running Copyright O 2015 Datera 61 LIO Admin Manual DATERA Tests such as starting stopping and suspending numbers of VMs were run with iometer workloads running on other VMs that weren t being started stopped or suspended Additional tests were run with all VMs running iometer and VMware snapshots were created and deleted as quickly as possible on all or some large subset of the VMs The results of these tests demonstrated that performance impact measured before or without VAAI was either eliminated or substantially reduced when using VAAI to the point that datastores could reliably be scaled to 128 VMs in a single LUN 11 5 Statistics The VMware esxtop command in ESX 5 has two new sets of counters for VAAI operations available under the disk device view Both sets of counters include the three VAAI key primitives To view VAAI statistics run esxtop from the VMware ESX 5 CLI and follow the steps below esxtop Pre
86. network Enable this attribute only if you are sure what you are doing see also Section 9 5 e login timeout integer Values 1 30 default value 15 Value in seconds to wait for an iSCSI login to complete before explicitly failing the login attempt e netif timeout integer Values 1 15 default value 2 Value in seconds to check for physical NIC link failure e prod mode write protect bool Values 0 1 default value O disabled Control for explicit initiator NodeACLs determining whether all associated LUNs should appear with write protect 1 Copyright O 2015 Datera 37 LIO Admin Manual MATERA 8 9 4 The parameter Config Group The following parameters are used to directly control iSCSI control properties per RFC 3720 see 6 AuthMethod enum Values CHAP None CHAP None Defines the authentication method used by the iSCSI target portal group TPG endpoint CHAP is the Challenge Handshake Authentication Protocol as defined in RFC 1994 see 17 DataDigest enum Values CRC32C None default value None Used to control the data digest usage by the iSCSI target portal group TPG endpoint RFC 3385 provides a discussion on the selection criteria for the CRC see 3 DataPDUInOrder bool iSCSI Default value Yes No is used by iSCSI to indicate that the data PDUs within sequences can be in any order Yes indicates that data PDUs within sequences have to be at continuo
87. nitiator with the ION ign 1991 05 com microsoft ibm t410s iscsi iqn 20 70c82fb tpgtl acls create iqn 1991 05 com microsoft ibm t410s Successfully created Node ACL for ign 1991 05 com microsoft ibm t410s Created mapped LUN 0 fi SeCsiL ky 20 5 s o 10602 t 5 Tage l This creates an iSCSI Node ACL with a mapped LUN O Copyright O 2015 Datera 47 LIO Admin Manual NATERA Node ACLs allow mappings of actual LUN IDs onto arbitrary Mapped_LUN IDs which are the LUN IDs presented to initiators These mappings can match preferred LUN IDs for particular initiators so for instance a LUN1 can be mapped onto Mapped LUNO to make LUN1 appear as iSCSI LUNO on the initiator Usually LUNs are identically mapped however Setup the CHAP logon information for an initiator which consists of the userid login name and password target secret from the initiator iscsi iqn 20 70c82fb tpgtl cd acls iqn 1991 05 com microsoft ibm t410s iscsi iqn 20 oft ibm t410s gt set auth userid iqn 1991 05 com microsoft ibm t410s Parameter userid is now iqn 1991 05 com microsoft ibm t410s iscsi iqn 20 0ft ibm t410s set auth password mytargetsecret Parameter password is now mytargetsecret iscsi iqn 20 oft ibm t410s gt get auth AUTH CONFIG GROUP mutual password mutual userid password mytargetsecret userid ign 1991 05 com microsoft ibm t410 iscsi iqn 20 oft ibm t410s gt cd iscsi figsegi gt The iSCSI Endpoint i
88. nstabilities and crashes For instance fast initialization of RAID devices from the LSI RAID BIOS creates seemingly valid RAID devices that can cause system crashes during operation under load In order to use a whole drive as the physical volume specify a raw disk device for lt device gt e g dev sdb This creates a volume group descriptor at the beginning of the drive dev sdb On the other hand to use a logical partition on a disk device specify the corresponding primary or extended partition for lt device gt e g dev sdb1 This creates a volume group descriptor at the beginning of the logical partition dev sdb1 Note LVM works fine with whole disk physical volumes but other OSs may not recognize the LVM metadata and display the drive as free and possibly overwrite it 4 3 Creating Volume Groups To create a volume group the following command is used vgcreate lt vg_name gt lt device gt 4 4 Creating Logical Volumes To create a logical volume within a volume group the following command can be used lvcreate L lt size gt name lt lv_name gt lt vg_name gt 4 5 Deleting Logical Volumes Warning Deleting a logical volume cannot be reverted All data on this logical volume will be lost Please make sure to unmount the logical volume before deleting it umount lt mount_point gt lvremove f lt vg_name gt lt lv_name gt 4 6 Resizing Logical Volumes If a logical volu
89. nt backstore plugin module 8 6 2 The attribute Config Group e block size integer Values 512 1024 2048 4096 The physical block size of the underlying device For IBLOCK and PSCSI backstores this is determined by the underlying physical device For FILEIO backstores this is a logical value e emulate tas bool Values 0 1 default value 1 enabled Control for emulated task aborted status This attribute determines whether the target will send a status response to the initiator upon task abort e emulate ua intlck ctrl bool Values 0 1 default value O disabled Control for unit attention interlock This attribute determines which unit attention to report when multiple unit attentions have been generated e enforce pr isids bool Values 0 1 default value 1 enabled Control for using persistent reservation with initiator session IDs ISIDS This attribute determines if ISIDS are included in the initiator port tuple iSCSI only e max sectors integer Values 1 hw max sectors default value hw max sectors The user settable max sectors for the backstore device Its value may be up to hw max sectors as reported by the backstore device e queue depth integer Values 1 hw queue depth default value 1 The user settable queue depth for the backstore device Its value may be up to hw queue depth as reported by the backstore device e task timeout integer Values 0
90. o 0 Storage Object backstores gt 9 3 iSCSI Endpoints Change to the top level iSCSI fabric object and instantiate an iSCSI target using a default IQN based on the system hostname and architecture backstores gt cd iscsi iscsi gt create Created target Lem 2005 01 ome later iscsi Saml il 004 ssm Sioxexe 1 IU R219 selected Mac Tag 1 SUECESSTVUL IN create TPE Io iscsi gt Targetcli per default automatically adds a Target Portal Group TPG and per default assigns a sequentially increasing TPG tag starting from 1 thereby adding a TPG 1 to the SCSI target Now add the backstores IBLOCK and FILEIO to the TPG iscsi ed iqn 2003 01 0rg linux iscsi san01 x8664 sn 35ee770c82fb tpgt1 iscsi iqn 20 70c82fb tpgtl luns create backstores iblock block backend Selected LUN 0 Successfully created LUN 0 iscsi iqn 20 70c82fb tpgtl luns create backstores fileio fileio backend selecites JUN 1 Copyright O 2015 Datera 45 LIO Admin Manual MATERA SUCESSEVIL IA crearss I N Lo lisesi iem 20 gt 7 tel 15 togti Targetcli per default automatically assigns sequentially increasing LUN numbers starting from 0 thereby exporting LUN O and LUN 1 in the example above The LUN ID can be explicitly set by using create with the lun 0 attribute Display the resulting iSCSI TPG d exea 3 gm s 20s 5 s 7068921tio tegtil Le ss icm 2003 01 Org Lime SAMO 9 664 cem ISS T 706882
91. o DISIDENTES noce eee ne ona ope dev vg_ssd lv_DBData03 activated O ido JURJDEWEENUA psp coc seda Apo poa dev vg_ssd lv_DBData04 activated Oc EE EE dev vg_ssd lv_DBData05 activated SE dev vg_ssd lv_DBData06 activated EE ore E ells dev vg hdd 1v DBFlashl activated os ado DELAS neces ogee kann neat oo dev vg hdd 1v DBFlash2 activated E T ET ETT dev vg ssd lv DBRedol activated o SDBROdO2 eL IT EN een oe dev vg ssd lv DBRedo2 activated 9 ilo DBVOESL cooocooocococooocoocooocoo dev vg ssd lv DBVotel activated om ID DBVOES2 e I ME E s dev vg ssd lv DBVote2 activated O allo DEWOIESS ooooc6600005ccnn0500060000c0 dev vg ssd lv DBVote3 activated SINS cT sqauninray asec dev vg hdd lv VSAL activated G ilo WSALWAREIMIWEMEEE nonnooooo dev vg hdd lv VSALArchiveMeta activated EECH dev vg_hdd lv_VSALMeta activated EE dev vg_hdd lv_VSALarch activated SS TIAS RRA eo C T terrier Peps Pepe GLO Or Io Hath or Pere Pe remem eee 0 Storage Object O wel CW C C TT 0 Storage Object O Bel MES nioo000000000050B009000500b009g090000004000900299 95 0 Storage Object O9 ilo SPPE oonosoboooocoooooooooon oooooooooocooononcooopbooocoooo onooop 0 Target oso MS ee ate Pere rere rene c TAO quete CT RETE TERR CO DOO EE 5 Targets O icm 200S35 01 exec s limtise i1lsesi seu 338664 8 Sim lecctolalao cossaaac LL EE O EPEL sane pa nana psi sus EE enabled OH EE asocoororoponoon doo svoneunoooonwote
92. on 9 9 1 Context History Using lt and gt for navigating the object context history ZS gt cd lt telalas you back to Aert A ica 2003 01 Ora y llaves iscsi san01 x8664 sn bf919196ff4e tpgtl iscsi iqn 20 196ff4e tpgtl ed gt Taking you back to Copyright 2015 Datera 52 LIO Admin Manual NATERA 9 9 2 Command History Using CTRL R bash style command history recollection note context sensitivity gt cd iscsi iscsi gt lt CTRL R gt reverse i search cd iqn cd iqn 2003 01 org linux iscsi san01 x8664 sn bf919196ff4e tpgtl iscsi ed iqn 2003 01 org linux iscsi san01 x8664 sn bf919196ff4e tpgt1 iscsi iqn 20 196ff4e tpgtl ed gt 9 9 3 TAB Completion Using TAB completion for directory objects gt cd lt TAB gt lt backstores iscsi qla2xxx gt last ib sico loopback path cacaos path keyword gt cd backstores lt TAB gt backstores fileio backstores pscsi backstores rd_mcp backstores iblock backstores rd dr 5 6 6 8 6 b path gt cd backstores iblock lt TAB gt block backend backstores i block backend Using TAB completion for object parameters The following example shows changing a per backend device attribute backstores i block backend set attribute lt TAB gt lolo Suse emfozes pr _ isicls emulate dpo max sectors EMMIS fue ee max unmap block desc count emulate tua write max
93. on in the LVM cache may arise such as duplicate physical volumes being displayed etc However disabling the LVM write setting becomes crucial in case of failover situations where stale information in the LVM cache can have serious adverse effects To disable the LVM write cache add the following entries into the file etc Ivm lvm conf write Cache state 0 readahead none Copyright 2015 Datera 12 LIO Admin Manual DATERA 5 Targetcli Quick Start Guide This section provides a simple step by step cook book to setup a simple LIO Linux SCSI target and export a Linux block device to which all initiators can connect without further authentication The same basic setup is described for iSCSI Fibre Channel and InfiniBand fabrics 5 1 Startup 5 1 1 Cookbook Command Comment targetcli Run the LIO admin shell backstores iblock create my_disk dev sdb Create a block backstore on dev sdb iSCSI follow Section 5 2 Fibre Channel follow Section 5 3 InfiniBand SRP follow Section O saveconfig Commit the configuration Table 1 Login and setup a block backstore 5 1 2 Invoke the LIO Shell The LIO shell is invoked by running targetcli as root from the underlying Linux shell targetcli Disken to The targeteli It Copre gmk E 2044 Jew Deireza Ime All riciicts reserved Visit us at http www datera io Using loopback fabric module Using ib srpt fabric module Using gla2xxx fabric module si
94. ons from the copyright owner of the materials to avoid violating the law which could results in serious damages and or remedies Datera Inc 2570 W El Camino Real Suite 380 Mountain View CA 94040 www datera io Phone 1 650 384 6284 Email info datera io Document Version 4 0 Print date 6 August 2015 Language English Copyright 2015 Datera i LIO Admin Manual NATERA Contents eli Meu 2 1 ntroductiOn eoo eerte S EE A DEREN NEUE NE E SENE EERO SENE ER EE PENNE NEU SENNEE T eENS ROSE N ER REPENE 1 2 OVONVIOW 2 2 1 The LIO LINUX Stack ee EE 2 Did e 2 23 Ee 65 ode ete thee rte etie eee tyi ee te ev evt ees ques fetenees diia 3 3 RAIDDewvice SetUp o ioneie oo ornato bona Iona yn n bona RR sts RR S S DRRR RR S SEE RRRR SS DERE RR SPORE EENS ENNA Naa 5 SIME TNAM I H 5 3 2 System Management eese Eee ene bones ce AE E E E desto vae V e ERE Eve 5 3 3 Creating a RAID Seti in cin i i ectetuer e Dt Fato du ca Eee Fo Fa toe a Ed cde 7 3 4 Deleting a RAID R EE 8 3 5 Expanding a RAID titi ate 8 316 Failure Management A A ia 9 EN Best Practices asta A need ada Reena aaadded said eiaa 10 4 MINIM Setup WEE 11 4 1 TO 11 4 2 Initializing Disks or Disk Partitions eene nennen enne nennen nnn nnns 11 4 3 Creating Volume GrouUpS ice cte ooa ee n caen nod cen nio cae
95. ot gt will help to identify the drive in the chassis by blinking the corresponding activity LED Copyright 2015 Datera 8 LIO Admin Manual NATERA When the array and all drive members to be used for the array extension have been identified and only then perform the online capacity expansion or RAID level migration arcconf MODIFY lt controller gt FROM lt DRIVE gt TO MAX lt raid level gt lt channel gt lt port gt lt channel gt lt port gt 3 5 2 LSI Display the logical drive information on a RAID controller MegaCli LDInfo L lt enclosure slot gt a lt controller gt List the available disk drive slots and pair down the information a bit MegaCli PDList a lt controller gt egrep Adapter Enclosure Slot Inquiry Identify the slot numbers of the drive s egaCli PDLocate PhysDrv lt enclosure slot gt a lt controller gt will help to identify the drive in the chassis by blinking the corresponding activity LED When the array and all drive members to be used for the array extension have been identified and only then perform the online capacity expansion or RAID level migration MegaCli LDRecon Start r lt raid level gt Add Rmv PhysDrv lt enclosure slot al L lt drive gt a lt controller gt For instance perform an online capacity expansion or RAID level migration with the physical drive s in enclosure O slot
96. pleted by the storage controller itself directly to disk 11 4 3 VMFS datastore scalability Documentation from various sources including VMware professional services best practices has traditionally recommended 20 to 30 VMs per VMFS datastore and sometimes even fewer Documents for VMware Lab Manager suggest limiting the number of ESX servers in a cluster to eight These recommended limits are due in part to the effect of SCSI reservations on performance and reliability Extensive use of some features such as VMware snapshots and linked clones can trigger large numbers of VMFS metadata updates which require locking Before vSphere 4 1 reliable locks on smaller objects were obtained by briefly locking the entire LUN with a SCSI Persistent Reservation Any other server trying to access the LUN during the reservation would fail and would wait and retry up to 80 times by default This wait and retry added to perceived latency and reduced throughput in VMs In extreme cases if the other server exceeded the number of retries errors would be logged in the VMkernel logs and I Os could return as failures to the VM When all ESX servers sharing a datastore support VAAI ATS can eliminate SCSI Persistent Reservations at least reservations due to obtaining smaller locks The result is that datastores can be scaled to more VMs and attached servers than previously Datera has tested up to 128 VMs in a single VMFS datastore The number of VMs was limited i
97. pped OO uos ooa o doo ror 6 0 E 0 06 0 0 0 0 0 0 50 E 0 9 ooo o 6 lunO rw Qs AMS ona oo Gon oo UH onm e ODO 07 9 qo 09 9 9 a oppooogoseo06 1 LUN om lun ia e RET TIE iblock my disk dev sdb Copyright 2015 Datera 56 LIO Admin Manual MATERA 10 3 InfiniBand SRP 10 3 1 Setup Script The following Python code illustrates how to setup a basic SRP target and export a mapped LUN usr bin python InfiniBand setup script example with RTSlib Erom resili import Setup an IBLOCK backstore backstore IBlockBackstore 3 mode create AS so IBlockStorageObject backstore fioa dev fioa gen wwn True except backstore delete raise Create an IB target endpoint using an ib srpt WWPN Eege Helga Mica Vals supe target Target fabric 0x00000000000000000002c903000e8acd tpg TPG target 1 Export LUN 0 via the so StorageObject class uno too Low 0 so my Lum Setup the NodeACL for an IB initiator and create MappedLUN 0 node acl tpg node acl 0x00000000000000000002c903000e8acd mapped lun node acl mapped lun 0 0 False Note that while SRP TPGs are masked by targetcli they are not masked by RTSlib 10 3 2 Object Tree The resulting object tree looks as follows EE EES EE SE ee e EE 0 Storage Object EE 1 Storage Object O my CHES aasocooocoorooooanaaoorrooooo coo oa HHO OOOO dev sdb activated ES EE 0 Storage Object SO el EE no eier EE EE EE TEE
98. r configuration see Section 12 4 2 9 5 4 TPG Authentication Setting up authentication information for every single initiator separately can be cumbersome so targetcli provides the capability to define common login information for all Endpoints in a TPG Consequently all initiators connecting to that TPG can use the same login credentials Enable common TPG Authentication for all Endpoints in a TPG iscsi iqn 20 70c82fb tpgt1 gt iscsi iqn 2003 01 org linux iscsi san01 x8664 sn bf919196ff4e tgptl set attribute demo mode write protect 0 generate node acls 1 cache dynamic acls 1 Parameter demo mode write protect is now 0 Baramerer gerente mock acils 18 mow UI Buerg gailen cls 18 mow Vil lisesi iem 20 o so 7 0692 t5 iter 19 Setup the common TPG Authentication credentials for all Endpoints in a TPG which consists of e The userid login name and password target secret for the target e The userid mutual login name and password mutual initiator secret for the initiator iscsi iqn 20 70c82fb tpgtl set auth userid rts user Parameter userid is now rts user iscsi iqn 20 70c82fb tpgtl set auth password b492785e bc91 4710 Parane terip Svor ro 1S mew 9492798 5e loe9 4 TIO iscsi iqn 20 70c82fb tpgtl set auth userid mutual mutual rts user PEremerer usezio MUCK 18 moy Vmibebel mes useie o iscsi iqn 20 70c82fb tpgtl set auth password mutual aeae2e26 f043 42a7 Parameter password m
99. r unloading the target service as the target configuration will be rolled back to the last saved one Copyright 2015 Datera 22 LIO Admin Manual MATERA 6 Targetcli Concepts 6 1 General Targetcli is an interactive shell with which users can create delete and configure LIO storage objects The fundamental metaphor of targetcli are context objects which represent the target stack objects Context objects are unified in a single hierarchical object tree that reflects their logical structure and relations Context objects are named by their full path in the hierarchical object tree which allows addressing and navigating them Entering an object changes the current object context corresponding to its current working path which is depicted in the command prompt pwd displays the complete current path e g if the prompt displays an abbreviated path for space efficiency cd navigates the object tree Without parameters cd presents the full objects tree The destination path can be selected via cursor keys help cd lists context sensitive navigation tips Due to the hierarchical structure of a Target object tree multiple context changes might be required to enter and traverse objects 6 2 Working with Contexts create context context name or short Zcontext name saves the current context under name which simplifies traversal of the object tree Saved contexts can be restored at any time and remain persistent across
100. re Name VMFS FIO1 Agent VM Settings Paths K Total Formatted Capacity 149 75 GB rmatting A Total 1 ae l File System VMFS 5 54 oS Block Size 1MB Disabled 0 di Name Target or Status contains y Clear X I Recent Tasks Name Target Status Tnitiated by Requested Start Ti Start Time Completed Time T Y RescanVMFS H 192 168 1 30 Completed System 8 8 2012 1 05 10 AM 8 8 2012 1 05 10 AM_ 8 8 2012 1 05 17 AM 3 DN Refresh host storages a 192 168 1 28 Completed MarcFleischmann 8 8 2012 1 04 51 AM 8 8 2012 1 04 51 AM 8 8 2012 1 05 03 AM J E CreateVMFSdatastore E 192 168 128 Completed MarcFleischmann 8 8 2012 1 04 42 AM 8 8 2012 1 04 42AM 8 8 2012 1 05 10 AM Z Tasks Alarms amp O Evaluation Mode 46 days remaining Marc Fleischmann gt gt d Figure 13 vSphere 5 the Storage View 12 5 Linux iSCSI Initiator A Linux iSCSI initiator can be setup as follows to use your Linux array This example is based on RHEL First install the Linux iSCSI initiator iscsi initiator utils with yum install iscsi initiator utils Determine the SCSI ION of the client to be used for ACL creation on the iSCSI target cat etc iscsi initiatorname iscsi Start the iSCSI service daemon service iscsi start Discover the available iSCSI targets 7 ees taucht m chisiowiesy t sendtargets 192 5151 59 Login to the iSCSI target iscsiadm 1
101. ret mutual userid mymutualuid mutual password mymutualsecret Parameter password is now mytargetsecret Parameter userid is now mytargetuid Parameter mutual password is now mymutualsecret Parameter mutual userid is now mymutualuid Parameter enable is now 1 iscsi get discovery auth enable 1 mutual password mymutualsecret mutual userid mymutualuid password mytargetsecret userid mytargetuid iscsi gt Only SCSI initiators that can authenticate themselves with a user id of mytargetuid and a password of mytargetsecret can now discover this iSCSI target and conversely the iSCSI target can only discover SCSI initiators that can authenticate themselves with a user id of mymutualuid and a password of mymutualsecret 9 7 Object Tree Display the resulting object tree from the top level root object isesi gt ls Om J EM TN G INECISTOLSS cnnnwmsonsoooooooon005090909009090009900209 58D DDOooD0000004d0089 0 0 000 SS aab Sao ec pure rere pian EE E IC PEE B Gero EE 1 Storage Object t etico backend ono ooo o ooo oboe e cM E usr sre iileio asitiveiteo I S RL T EE 1 Storage Object ES Dlock backend rt EE dev sdb activated O ONE 1 Storage Object MOS T A ocooococococonaaooocoooocooccoccotnaooobb5co dey sr0 activated cuc Ncc DDD OOD OD DOOD ODDO DODD OODODDDDOD 0 Storage Object EE SS socccotor ory o 0 00 6 66 Co dur Ou0 0 9 0 0 0 0 6 bebo 6706 16 979 8 0 Storage Object
102. roller alert notifications can be configured LSI Mert Settings mad Server Emad You may edt the alert deleery methods for each event severty level Al events at that severity level wil use the listed alert delivery methods unless you define exceptions by changing individual events Alert Debvery Methods System Loo MSM Log Popup System Log MSM Log System Log MSM Log L9 wa A Figure 2 LSI MSM Configure Alerts For more information please refer to the corresponding LSI manual 8 3 3 Creating a RAID Set 3 3 1 Adaptec Before creating a new RAID set check which disk drive ports that should be used for the new RAID set arcconf GETCONFIG lt controller gt PD egrep Device State gt Reported Location Reported Channel S M A R T Warnings Copyright O 2015 Datera 7 LIO Admin Manual MATERA Create the new RAID set arcconf CREATE logicaldrive lt controller gt STRIPESIZE lt stripe gt MAX lt raid level gt lt channel gt lt slot gt 3 3 2 LSI List all physical disk details including enclosure number and slot numbers for a RAID controller MegaCli PDList a lt controller gt Create new RAID sets MegaCli CfgLDAdd r0 rl1 r5 r6 lt enclosure gt lt slot gt WT WB NORA RA Direct Cached CachedBadBBU NoCachedBadBBU strpsz lt mb gt Hsp lt enclosure gt lt drive gt a lt adapter gt For instance create a
103. round concepts tools and commands for effectively administrating data storage arrays running the LIO Linux SCSI Target and the targetcli system management tools The key steps to configure LIO based SANs are described in this LIO Admin Manual as follows Section 2 provides an overview over LIO Section 3 describes how to setup hardware RAID devices to use for your backstore Section 4 describes how to use and setup the Logical Volume Manager LVM for your backstore Section 5 provides a quick start tutorial how to set up Linux LIO SAN targets with targetcli Sections 6 9 provide an in depth technical description of targetcli Section 12 describes how to setup Microsoft Windows vSphereVMware Linux and VirtualBox iSCSI initiators for Linux LIO SANs Copyright 2015 Datera 1 LIO Admin Manual DATERA 2 Overview 2 1 The Linux Storage Stack The following diagram depicts the basic architecture of the Linux storage stack with LIO iSCSI TPG Target Portal Group TPG Target Portal Group TPG Target Portal Group LUN LUN LUN LUN T LUN LUN L LUN LUN T f LUN LUN T LCUN LUN Na BS iSCSI VG Volume Group VG Volume Group PV Physical Volume PV Physical Volume LVM SSD RAID 5 HDD RAID 10 HW RAID Controller Figure 1 Structural overview over SCSI entities and relationships BS means back store The LIO software stack is divided in two main sections software modules
104. rt a mapped LUN usr bin python Fibre Channel setup script example with RTSlib Erom elle import Setup an IBLOCK backstore backstore IBlockBackstore 3 mode create CEV so IBlockStorageObject backstore fioa dev fioa gen wwn True except backstore delete raise Create an FC target endpoint using a qla2xxx WWPN fabric FabricModule qla2xxx target Target fabric 21 00 00 24 ff 31 4c 48 tpg TPG target 1 Export LUN 0 via the so StorageObject class luno tpg lua 0 so my lua Setuj the NodeACiL o em INC imitiator anc Creatas MeEjoeslau 0 node el wee mocks ceil 2120080082481 SUL S dier die y mapped_lun node acl mapped lun 0 0 False Note that while Fibre Channel TPGs are masked by targetcli they are not masked by RTSlib 10 2 2 Object Tree The resulting object tree looks as follows RS UT O JH SEBES EE EE EE EE EE E oo aora ee O EE 0 Storage Object EE EE 1 Storage Object G my CHS asoocosoccoroooonaaoorsoroo oro E ooo dev sdb activated ES O 0 Storage Object EEN 0 Storage Object G il mE oo605o0co0c0000006900000000000000000005006000090000 0 Storage Object BA NI sora rro pophoooodo don EDD DD 0 0 o oa Hx Ora D EID po bo add ooo 0 Target ES cdd Lu c E 0 Target EE 0 Target Oi AG PINS Wess EE 3L target o E O EE enabled G GOLS cusmtu5 p b b b D o C00 EELER EC 0 ACL SS E EH cS a e e S SESS SET T TOMUS 1 Mapped LUN o ma
105. s The Initiator or target declare the maximum data segment length in bytes they each can receive in an iSCSI PDU The transmitter initiator or target is required to send PDUs with a data segment that does not exceed MaxRecvDataSegmentLength of the respective receiver A target receiver is additionally limited by MaxBurstLength for solicited data and FirstBurstLength for unsolicited data An initiator must neither send solicited PDUs exceeding MaxBurstLength nor unsolicited PDUs exceeding FirstBurstLength or FirstBurstLength minus lt Immediate Data Length gt if immediate data were sent OFMarkint lt integer gt Values 0 65535 default value 2048 Used to set the interval for the initiator to target markers on the connection OFMarker bool iSCSI Default value No Used to control the usage or the initiator to target markers on the connection TargetAlias lt string gt If a target has been configured with a human readable name or description this name should be communicated to the initiator during a Login Response PDU if SessionType Normal This string is not used as an identifier nor is it meant to be used for authentication or security purposes It can be displayed by the initiator s user interface in a list of targets to which it is connected The parameter Config Group This group s parameters are specific to each fabric module If no parameters are available for a fabric module this group is omitted 8
106. s the SCSI Network Portal and automatically changes the working context to it Return to the underlying TPG as no attributes need to be set for a standard iSCSI Network Portal Jigsesa 16 20 68 1 139332605 eel lt Taking you back to iscsi iqn 2003 01 org linux iscsi targetcli demo x8664 sn 05135a0e4all tpgtl iscsi iqn 20 a0e4a11 tpgti1 5 2 3 Export LUNs via iSCSI Declare a LUN for the backstore device to form an Endpoint a valid network storage object iscsi iqn 20 a0e4all tpgtl luns create backstores iblock my disk Selected LUN 0 Successfully created LUN 0 Entering new node iscsi iqn 2003 01 0rg linux iscsi targetcli demo x8664 sn 05135a0e4a11 tpgt1 luns lun0 lisesl iem 20 s e T IT tuns Targetcli automatically assigns the default ID 0 to the LUN and then changes the working context to the Endpoint Now the target is created and exports the local device dev sdb as SCSI LUN 0 Return to the underlying TPG as no attributes need to be set or modified for standard LUNs iscsi iqn 20 gtl luns lun0 gt ed lt Taking you back to iscsi iqn 2003 01 0rg linux iscsi targetcli demo x8664 sn 05135a0e4all tpgtl iscsi iqn 20 a0e4a11 tpgt1 5 2 4 Define Access Control Configure access control Typically this involves setting up ACLs with individual login information for each initiator For a simple demo setup allow access to all initiators without any authentication iscsi iqn 20
107. s to ready for secure logins from the specified iSCSI initiator Note The Microsoft Windows iSCSI Initiator uses its IQN as a default login name It requires the password length to be at least 12 bytes 96 bits and it rejects passwords that are too simple The Microsoft Windows iSCSI initiator allow changing the default login name through the Targets tab gt Connect button gt Connect To Target dialog gt Advanced button gt Advanced Settings dialog gt Enable CHAP log on checkbox The resulting two iSCSI targets one with ACLs for CHAP initiator authentication and one in Demo Mode look as follows Jisesi gt Ts Ong Als CS O eee EE EE 2 Targets Q acm 2003 01 seg limib lt 1Ssesi seadll 00043 eua 3 5xexe 7 VOCS ES oooonocooao LL TEE IEJOJO REA Bonn nanos o Oooh PS SAMOS OOO 0 0 D C OAR AOD OOO Oo oboe ERROR enabled e Hels op 0 o00000000000000000000000000000000 0000099 OHHOOOHOD OOS EEOS 1 ACL o cm 1991 05 Com HEI Mom c41OS sosooooooococooocoaoo 2 Mapped LUNs SMS mimmo cogo c domos 9 099000009 009000909 0999990645 lunO rw G mappse lumi ooosoccocoonoooccoooooooocooonnnoooooooocoo oooo lunl rw OF beet o raras cones EE 6 C O GG OC O O O 0 6 0 0 6 6 EE 6 a8 2 LUNS ESO Fee e Mt P iblock block backend dev sdb OS MMI 556ooo002000202000000c00 Filelio filelo backend Jussit lei O portals eere EE OI 3L Portal Om 192 ROS Oe lS SZ OO EE OK O auem 2009 01 org limu 18 es exe
108. sessions enter context context name or short 9context name restores the corresponding saved context name and enters it at the same time Saved contexts allow both naming and bookmarking available transport objects Each context objects provides context sensitive operations i e different context objects or paths provide different command sets For instance a path pointing at an iSCSI target provides different commands than a path pointing at a storage object 6 3 Command Completion and Help At any time in the targetcli shell TAB triggers command auto completion With a unique option pressing TAB once auto completes the current command being edited With multiple options hitting TAB twice produces a list of available command completions if any and a quick syntax help on the command currently being edited This is useful when the exact next parameters available for the current command are not known Each command parameter can be passed either as a positional parameter in order of the command syntax or as a key value pair in any order Command auto completion will reflect and present all available options To list a full command syntax and description help command Copyright O 2015 Datera 23 LIO Admin Manual MATERA 6 4 Command Syntax lt path gt lt command gt lt parameters gt lt path gt provides an optional path name or context in which the lt command gt is executed command is run
109. ss u to change to the disk device stats view 2 Press f to select fields or o to change their order Note This selects sets of counters not individual counters 3 Press o to select VAAI Stats and or p to select VAAI Latency Stats Optionally deselect Queue Stats I O Stats and Overall Latency Stats by pressing f g and respectively in order to simplify the display ay l 5 To see the whole LUN field widen it by pressing L capital then entering a number 36 is wide enough to see a full NAA ID of a LUN The output of esxtop looks similar to the following 4 46 50am up 44 min 281 worlds 0 VMs 0 vCPUs CPU load average 0 00 0 00 0 00 DEVICE CLONE RD CLONE WR CLONE F MBC RD s MBC WR s ATS ATSF ZERO ZE RO F MBZERO s DELETE DELETE F MBDEL s naa 60014050e4485b9bdc841d09478888e6 0 0 0 0 00 0 00 23 0 0 0 0 00 0 0 0 00 naa 600140515743d5195b04 98b8aad6 fdd2 1583 792 0 0 00 0 00 1322 0 23 0 0 00 0 0 0 00 naa 60014053937c69d44 4e0b9e5a95398 0 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 naa 60014055fc 891d0c5b4a60a66942400 4746 3955 0 0 00 0 00 4402 0 45 0 0 00 0 0 0 00 naa 600140573d94 8e531d4d1ab5c8a72ef 0 0 0 0 00 0 00 23 0 0 0 0 00 0 0 0 00 naa 6001405a2e547c17329487b865dla66e 3164 4746 0 0 00 0 00 5692 0 54 0 0 00 0 0 0 00 naa 6001405a3a17fe4483c46f994 74b4e6 0 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 t10 ATA T3400832AS 0 0 0 0 00 0 00 0 0 0 0 0 00 0 0 0 00 The VAAI
110. ted and exports dev sdb as LUN O Copyright O 2015 Datera 16 LIO Admin Manual MATERA Return to the underlying Endpoint as no attributes need to be set or modified for standard LUNs Jede isses BOUL 0 6 oo 9 9 lias E wins crcl lt lt Taking you back to qla2xxx 21 00 00 24 ff 31 4c 48 ars d Seel a 5 3 4 Define Access Rights Configure the access rights This involves setting up individual access rights for each initiator based on its WWPN First determine the WWPN for the respective Fibre Channel initiator For instance for Linux initiator systems use cat sys class fc host host port name sed e s 0x e s amp g e s For a simple setup allow access to the initiator with the WWPN as determined above qla2xxx 21 0 4 f 31 4c 48 acls create 21 00 00 24 ff 31 4c 4c Successfully created Node ACL for 21 00 00 24 ff 31 4c 4c Created mapped LUN 0 Entering new node qla2xxx 21 00 00 24 ff 31 4c 48 ac1s 21 00 00 24 ff 31 4c 4c Zeile d St ele EE Ae Ae cel A ZS The targetcli shell then automatically adds the appropriate mapped LUNs per default 5 3 5 Display the Object Hierarchy The resulting Fibre Channel SAN object hierarchy looks as follows displayed from the root object gt 1s EE O JEE kE COCOS 6865 E EE o Oo OOO 0 00 ENEE LEE SE e EE 0 Storage Object Io UNS aa EE EE E ee 1 Storage Object IPOS my disk aaa o arie E E EEE e dev sdb activated SS EE
111. the following commands are equivalent e cd mybookmark e bookmarks go mybookmark Bookmarks can be used anywhere for a path and can be conceptualized as a variable for that path e mybookmark Is performs the Is command in the bookmarked path e Is mybookmark shows the object tree beneath the bookmarked path action can be either e add adds the current path to the bookmarks e del deletes a bookmark e go navigates to a bookmarked path e show shows all bookmarks bookmark is an optional parameter that specifies the name of the bookmark 7 2 The cd Command cd lt path gt Change current working path or context to path path is constructed like a Unix path as separator for the current node for the parent node Example Assume the following a object tree a0 1 b0 Zi EL 3 SI H eo eO 2 Assume that the current node is the one marked by above for all following examples e cd navigate to node 1 e cd don t change the current object neutral operation e cd ai b0 c0 do navigate to node 2 s cd al navigate to node 3 e cd al navigate to node 3 too e cd navigates to the root node Copyright O 2015 Datera 28 LIO Admin Manual MATERA e cd a0 b0 c0 a1 navigates to node 3 The path history can be traversed with lt and gt e cd navigates one step back in the path history inverse operatio
112. tion please see also the Windows SCSI Initiator manual 9 12 2 3 Configuring LUNs Block devices can be configured with the Disk Management snap in which is part of the Computer Management console in the Administrative Tools Go to Start Control Panel search for Create and format hard disk partitions and start the Disk Management snap in The Disk Management window is in four parts On the left is the Computer Management pane of the Computer Management console The lower pane contains the information for all the block devices The upper pane contains descriptions of those drives including data on the amount of space used and free the type of file system and the health of that system The right pane contains a list of actions you can perform The list changes based on the object you select To use Disk Management to create or delete partitions format drives and create striped or volume sets click the drive that you want to modify and select the change that you want to make from the Action menu or Actions pane Alternatively you can right click on the drive that you want to modify and then select the appropriate action from the context menu Copyright O 2015 Datera 66 DATERA LIO Admin Manual Computer Management 70000 i Sk les ES File Action View Help e 2 Computer Management Local Status Volume Layout Type File System 4 System Tools Simple Basic Hea
113. traffic across RDMA InfiniBand interconnects It has traditionally been used in high performance computing HPC environments and recently has enjoyed increasing popularity for SANs LIO supports SRP on Host Channel Adapters HCAs from Mellanox e Mellanox ConnectX 2 VPI PCle Gen2 HCAs x8 lanes single dual port QDR 40 Gb s IB e Mellanox ConnectX 3 VPI PCle Gen3 HCAs x8 lanes single dual port FDR 56 Gb s IB e Mellanox ConnectX IB PCle Gen3 HCAs x16 lanes single dual port FDR 56 Gb s IB 2 2 5 InfiniBand iSER SER is a high performance communication protocol that extends iSCSI to use RDMA RDMA has been supported on InfiniBand networks and is now also supported by RoCE on lossless Ethernet networks and by iWARP enhanced TOE NICs over standard Ethernet networks LIO supports iSER on Host Channel Adapters HCAs from Mellanox e Mellanox ConnectX 2 VPI PCle Gen2 HCAs x8 lanes single dual port QDR 40 Gb s IB e Mellanox ConnectX 3 VPI PCle Gen3 HCAs x8 lanes single dual port FDR 56 Gb s IB e Mellanox ConnectX IB PCle Gen3 HCAs x16 lanes single dual port FDR 56 Gb s IB 2 2 6 Loopback The loopback fabric module is a high speed SCSI emulation device that can export any type of raw hardware to local applications and virtual machines as a fully SCSI SPC 3 4 compliant block device including emulation of advanced functionality 2 2 7 vHost The LIO vHost fabric module implements very high speed SCSI I O processing based on the Linux
114. uch as for Persistent Reservations PR or ALUA as used e g by VMware vSphere are typically not implemented in the storage device firmware and can thus cause severe malfunction of the storage array Instead use IBLOCK for such production setups 2 3 4 Memory Mapped RAMDISK The memory mapped RAM disk rd mcp devices provide RAM disk drive based SCSI emulation It has multi session capability as it supports separate memory mappings for initiators using memory copy rd mcp is a backend that is most useful for ultra fast volatile mass storage such as with HPC environments that need maximum performance storage for volatile data hot spots Copyright O 2015 Datera 4 LIO Admin Manual NATERA 3 RAID Device Setup 3 1 Overview 3 1 1 RAID Controllers Linux typically uses RAID volumes for its backstore storage media which can be implemented with hardware or software RAID Hardware based RAID controllers include Adaptec and LSI MegaRAID RAID controllers Adaptec RAID controllers can be configured via arcconf Adaptec s CLI for RAID devices or via the Adaptec Storage Manager ASM LSI MegaRAID controllers can be configured via MegacCLI LSI s CLI for RAID devices or via the graphical LSI MegaRAID Storage Manager MSM For Adaptec run arcconf as follows arcconf options For LSI run the MegaCL as follows MegaCli options For more information please also refer to the Adaptec manuals 1 and 2 or the LSI manuals 7
115. un 66643 sm lor 9191 9611479 ci osoocaaaao LL TEE O POG ee ao aaa EE E TT suai enabled EEN 0 ACL LUE E A A 1 LUN aia pscsi pscsi backend dev sr0 O ONS CARES mapas IP e cad dL Borra O A EE OK iscsi gt For configuring explicit Node ACL authentication see Section 8 13 3 For the corresponding Microsoft Windows iSCSI initiator configuration see Section 12 2 2 and for the corresponding VMware vSphere SCSI initiator configuration see Section 12 4 2 Copyright O 2015 Datera 48 LIO Admin Manual MATERA 9 5 3 Mutual CHAP Authentication Enable secure sessions for the initiator with the ION iqn 1991 05 com microsoft ibm t410 iscsi iqn 20 70c82fb tpgtl acls create iqn 1991 05 com microsoft ibm t410s Successfully created Node ACL for iqn 1991 05 com microsoft ibm t410s Created mapped LUN 0 lisesl iem 20 gt tel Ee Togs This creates an iSCSI Node ACL with a mapped LUN O Node ACLs allow mappings of actual LUN IDs onto arbitrary Mapped LUN IDs which are the LUN IDs presented to initiators These mappings can match preferred LUN IDs for particular initiators so for instance a LUN1 can be mapped onto Mapped LUNO to make LUN1 appear as iSCSI LUNO on the initiator Usually LUNs are identically mapped however Setup the Mutual CHAP logon information for an initiator which consists of e The userid login name and password target secret for the target e The mutual userid login name and mutual
116. unmap lba count emulare tas optimas EE emulate tpu queue depth mulate tpws task timeout emulare we lmelels ctri unmap granularity mulate write cache unmap granularity alignment SEELEN S RUSSIE S gab RU keyword backstores i block backend set attribute emulate write cache 1 SETE STE molen Wiles eine 18 mow Vil backstores i block backend 9 10 Bookmarks Bookmark the second IQN as my second target iscsi iqn 20 196ff4e tpgtl bookmarks add my second target Bookmarked iscsi iqn 2003 01 0rg linux iscsi san01 x8664 sn bf919196ff4e tpgtl as my second target iBesi icm 20 s Jd S aede e Eeer Illustrate using 0mmy second target bookmark and change directory from the top level root context iscsi ign 20 196ff4e tpgtl cd gt ls my second target r EI e T a O E O S E a a a A N A OA enabled o acls goros CC IU IEEE SSE E SS 0 ACL ENEE EE 1 LUN O EE pscsi pscsi backend dev sr0 NS E 3L Ported EE OK Copyright O 2015 Datera 53 DATERA LIO Admin Manual gt ed my second target iscsi iqn 20 196ff4e tpgt1 gt Create a bookmark for the first iSCSI target ION from the top level root object iscsi iqn 20 196ff4e tpgtl cd gt iscsi iqn 2003 01 org linux iscsi san01 x8664 sn 35ee770c82fb tpgt1 bookmarks add my first target Bookmarked iscsi iqn 2003 01 0rg linux iscsi san01 x8664 sn 35ee770c82fb tpgtl as my eeng terget gt
117. usly increasing addresses and overlays are prohibited DataSequencelnOrder bool iSCSI Default value Yes A Data Sequence is a sequence of Data In or Data Out PDUs that end with a Data In or Data Out PDU with the F bit set to 1 A Data Out sequence is sent either unsolicited or in response to an R2T Sequences cover an offset range Yes indicates that data sequences must be transferred using continuously non decreasing sequence offsets R2T buffer offset for writes or the smallest SCSI Data In buffer offset within a read data sequence A target may retry at most the last R2T and an initiator may request at most retransmission for the last read data sequence For this reason if DataSequencelnOrder is Yes and ErrorRecoveryLevel is not 0 then MaxOustandingR2T must be set to 1 No indicates that Data PDU sequences may be transferred in any order DefaultTime2Retain lt integer gt Values 0 3600 default value 20 The initiator and target negotiate the maximum time in seconds after an initial wait Time2Wait before which an active task reassignment is still possible after an unexpected connection termination or a connection reset This value is also the session state timeout if the connection in question is the last LOGGED_IN connection in the session IO indicates that connection task state is immediately discarded by the target DefaultTime2Wait integer Values 0 3600 default value 2 The initiator
118. utual is now aeae2e26 f043 42a7 3 exeat item 20 5 s 70692 Ee tpati gt Note Login credentials for specific initiators can be created by adding corresponding ACL entries as individual ACL entries override common TPG Authentication information 9 6 Discovery Control The iSCSI protocol can also control the visibility of iSCSI targets for discovery by SCSI initiators 9 6 1 CHAP Initiator Discovery Authentication Enable CHAP initiator discovery authentication for all initiators by setting up a CHAP userid login name and password target secret in the global discovery auth group Copyright O 2015 Datera 50 LIO Admin Manual NATERA iscsi gt set discovery auth enable 1 userid mytargetuid password mytargetsecret Parameter enable is now 1 Parameter password is now mytargetsecret Parameter userid is now mytargetuid a exea Only SCSI initiators that can authenticate themselves with a user id of mytargetuid and a password of mytargetsecret can now discover this iSCSI target 9 6 2 Mutual CHAP Discovery Authentication Enable Mutual CHAP discovery authentication for all initiators by setting up Mutual CHAP information in the global discovery auth group e The userid login name and password target secret for the target e The mutual userid login name and mutual password initiator secret for initiators iscsi set discovery auth enable 1 userid mytargetuid password mytargetsec
119. virtio mechanism LIO vHost provides virtually bare metal local SCSI storage performance for KVM guests Linux guest VMs are supported while Windows guest support is currently in Alpha release stage with a virtual LSI MegaRAID SAS driver 2 3 Backstores LIO supports the following backstores physical or virtual block storage devices for export over any of the above fabrics via the LIO Linux SCSI Target 2 3 1 FILEIO FILEIO is any file on a mounted filesystem which may be backed by a file or an underlying real block device FILEO is recommended mostly for testing purposes as the performance bottleneck with a block device residing in a filesystem is limited to the filesystem performance envelope Copyright O 2015 Datera 3 LIO Admin Manual NATERA 2 3 2 IBLOCK IBLOCK is any block device visible to LIO to be exported for native block based device access LVM represents its logical volumes as block devices and therefore is the recommended approach for exporting block devices 2 3 3 PSCSI The PSCSI backstore device provides pass through SCSI for any SCSI storage backstore object without SCSI emulation PSCSI does not contain any emulation and this exposes SCSI devices natively This provides maximum performance for such SCSI backstores but at the same time minimizes architectural layering and device encapsulation Warning Do not use PSCSI backstore devices unless you know exactly how they will be used Advanced SCSI CDBs s
120. w click the Targets tab A list of the targets and their storage devices appears To connect to a specific target select it and click Connect The Connect To Target dialog appears Select to Add this connection to the list of Favorite Targets to automatically restore the connection during system startup and optionally select to also Enable multipath Copyright O 2015 Datera 64 DATERA LIO Admin Manual Targets Discovery Favorite Targets Volumes and Devices RADIUS Configuration Quick Connect To discover and log on to a target using a basic connection type the IP address or DNS name of the target and then click Quick Connect Target Quick Connect Discovered targets Refresh Name Status e E Connect To Target A a o ign 2003 01 org linux iscsi mercury x8664 sn e06bdeb8 Inactive ign 2003 0 1 0rg linux iscsi node 1 x8664 sn a4eeb242b Reconnecting Target name ign 2003 0 1 org linux iscsi rtsnode 1 x8664 sn 1537795 Reconnecting qn 2003 0 1 org linux iscsi mercury x8664 sn ce0e83000332 v Add this connection to the list of Favorite Targets This will make the system automatically attempt to restore the connection every time this computer restarts To connect using advanced options select a target and then Paa Enable multi path dick Connect K To completely disconnect a target select the target and

Administrator`s Manual

Contents

Download Pdf Manuals

Related Search

Related Contents