Home

Monitoring the AWS EC2 Cloud

1. 19 MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances
2. MONITORING THE AWS EC2 REGION Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in
3. 10 MONITORING THE AWS EC2 CLOUD 10 11 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG system embeds an optional detailed diagnostic capability With this capabi
4. button to move the test to the ENABLED TESTS list Finally click the Update button Purpose Powers a specified VM on and off at configured intervals and in the process verifies the success failure of the corresponding operation i e powering on off and also reports the time taken by that instance to power on and off Failed attempts to power on and significant delays in powering on are thus brought to light Target of the Amazon EC2 Region test Agent A remote agent deploying the test 50 MONITORING THE AWS EC2 REGION Configurable parameters for the test 10 TEST PERIOD How often should the test be executed by default this is set to 24 Ars HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding s
5. alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site 7 EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 57 MONITORING THE AWS EC2 REGION 10 11 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy serve
6. operations Indicates the rate at which disk write operations were performed on this instance These measures are good indicators of the level of disk I O activity on an instance By comparing the values of these measures across instances you can accurately determine which instance is performing I O intensive operations 2 4 3 AWS EC2 Instance Uptime Test In cloud based environments it is essential to monitor the uptime of server instances launched on the cloud By tracking the uptime of each of the instances administrators can determine what percentage of time an instance has been up Comparing this value with service level targets administrators can determine the most trouble prone areas of the infrastructure hosted on the cloud In some environments administrators may schedule periodic reboots of their instances By knowing that a specific instance has been up for an unusually long time an administrator may come to know that the scheduled reboot task is not working on an instance This test monitors the uptime of each instance available to the configured AWS user account Purpose Monitors the uptime of each instance available to the configured AWS user account Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test 35 MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The
7. IT resources to deal with changes in requirements or spikes in popularity related to your application or service With many mission critical applications now being delivered via the cloud end users have come to expect from the cloud the same quality of service that local service deployments are known to deliver This means that even the slightest dip in performance levels will not be tolerated A sudden non availability of the cloud no matter how brief or a slowdown failure of any of its regions availability zones instances can make it impossible for cloud providers to build and launch mission critical services on the cloud and for consumers to access these services for prolonged periods If you are a public or private cloud service provider therefore your primary concerns would be can people access my service Is the self service portal up Can users see their VMs Can users connect to their VMs If not you need to be able to determine why the problem is happening is it the web front end is it due to the virtualization platform is it due to the SAN etc The action you take depends on what you diagnose as being the root cause of the problem Besides problem diagnosis you are also interested in understanding how you can get more out of your current cloud investments You want to be able to see how to balance load across your servers to serve a maximum number of users and how you can optimize the capacity of the infrastructure without sacrif
8. To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for each type of instance launched by the configured AWS user account in the test monitored region Measurement Measurement Measurement Interpretation made by the Unit test CPU utilization Percent A high value for this measure indicates that Indicates the one more instances of a type are utilizing CPU percentage of excessively this could be because of one more allocated CPU resource intensive processes executing on the consu
9. a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics 4 CONFIRM PASSWORD Confirm the password by retyping it here 5 AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account 6 CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure
10. access the server instances launched in that region This may in turn adversely impact the user experience with the cloud To avoid such an unpleasant outcome it is best to periodically monitor the availability of each region so that unavailable regions can be quickly and accurately identified and the reasons for their non availability remedied This test performs periodic availability checks on the monitored region and reports the status of that region In addition the test also indicates the time taken for connecting to the region so that connectivity issues can be isolated Purpose Performs periodic availability checks on the monitored region and reports the status of that region In addition the test also indicates the time taken for connecting to the region so that connectivity issues can be isolated Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test 44 MONITORING THE AWS EC2 REGION Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS a
11. agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG suite embeds an optional detailed diagnostic capability With this capability the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a particular server choose the On option To disable the capability click on th
12. aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enable
13. epe as est obla e obese bate ep Pix aL ek quide 15 2 3 The AWS Cloud Instance Status L yet i iie e rerba eee eR PENNE SEVEE AVE ERU XERE YER ERE EXE RESTE NE EATE TERRE 18 2 3 1 AWS EC2 Instance Connectivity Test eeeeeseeeeeeeeeeeee nennen ennt etn ne nnne nne nne nenne nenne tnnt enne nenne 19 2 3 2 LwdseeMnrue agc m M 22 2 4 The AWS Cloud Instance Details Layer essent nennen eren epai ened eanais erpe iaer 25 2 4 1 AWS BC2 Aggregated Resource Usage Test ic cccsscsscccsssssscsssseccecsscsscnsesosnoicvesdevtsonnssnegseusecssssissosevscdoecsesssserecesdeoes 25 2 4 2 AWS EC2 Instance Resources Test iei eene rtr ttn tote orbe eo eb uS EET e ESEESE Y SERES EHE NEUES REESE T SNES TIERS 31 2 4 3 AWS EC2 Inst nce Uptime Lest virus oen prt tr i pre ined ESAE Ta EER o ieSe era Ens area e e ia eesi 35 MONITORING THE AWS ECZ REGION eere esee ente sets tasas tato stata stata euis esso s tasa sea ease deesses aissos ondare seos asds teeside ss 39 3 1 The AWS EC2 Region Status Layer indir ter ier ere e e Biete n ga tese iie tea bee e o FE e ege 41 3 1 1 BC2 Availability Zones Test rid ene eie pier edi bep ir Edi 41 3 1 2 ECZ Rerions TeSt srar 44 3 1 3 AWS EC2 Web Access Fest M 46 32 The AWS EC2 Region Instance Stat s Layer 4 oie cer aA E ESNE ENS ELETE E NERVES PE HE TE RENE RE Rege
14. host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configur
15. launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to rue On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 20 MONITORING THE AWS EC2 CLOUD 10 PROXYHOST and PROXY PORT In some environments
16. longest to connect to 2 3 The AWS Cloud Instance Status Layer The tests mapped to this layer take stock of the total number of instances that are available for the configured AWS user account on the cloud and points you to the following e The powered off instances e The newly launched removed instances e Instances that are unavailable over the network 18 MONITORING THE AWS EC2 CLOUD 2 3 1 est AWS Cloud Instance Status Search sd eo IV All Ed AWS EC2 Instance Connectivity Wi ap southeast 1 karthika i bOc3efe2 E us east 1 karthika i O7dbe56d f AWS EC2 Instances Figure 2 4 The tests mapped tothe AWS Cloud VM Status layer AWS EC2 Instance Connectivity Test Sometimes an instance could be in a powered on state but the failure of the operating system or any fatal error in internal operations of the instance could have rendered the instance inaccessible to users In order to enable you to promptly detect such hidden anomalies this test periodically runs a connectivity check on each instance available for the configured AWS user account and reports whether the instances are accessible over the network or not Purpose Runs a connectivity check on each instance available for the configured AWS user account and reports whether the instances are accessible over the network or not Target of the An AWS EC2 Cloud test Agent A remote agent deploying the test
17. mware i bOc3efe4 ami 93ec93c3 windows mi small ap southeast ias enabled zap_db i bOc3efe7 ami 93ec893c6 Ny vindovs mi small ap southeast ia enabled zap mware i bOc3efe8 ami 93ec833c7 windows mi small ap southesst is enabled Figure 3 7 The detailed diagnosis of the Instances powered off measure 3 3 The AWS EC2 Region Instance Details Layer The tests mapped to this layer auto discover the server instances that are available for the configured AWS user account in a region and reports the uptime and the resource usage of the individual instances Resource hungry instances and those that were recently rebooted can thus be isolated 60 MONITORING THE AWS EC2 REGION AWS EC2 Region Instance Details search O U e EC2 Instance Resources E zap db i bOc3efe7 E zap mware i bOc3efe4 zap mware i bOc3efe8 zap db i bOc3efe3 zap db i bOc3efes5 zap_mware i bOc3efe2 zap_mware i bOc3efe6 Be EC2 Aggregated Resource Usage B mi small We EC2 Instance Uptime B zap db i bOc3efe3 E zap db i bOc3efe5 B zap mware i bO c3efe2 B zap mwars i bOc3efe6 Figure 3 8 The tests mapped to the AWS EC2 Region Instance Details layer 3 3 1 X EC2 Aggregated Resource Usage Test When users launch an instance using the AWS management console they need to specify the instance type An instance type is a specification that defines the memory CPU storage capacity and hourly cost for an instance Some instance types are designed fo
18. service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 64 MONITORING THE AWS EC2 REGION 10 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics
19. site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests In the EXCLUDE INSTANCE text box provide a comma separated list of instance names or instance name patterns that you do not wish to monitor For example b0c3e 7dbe56d By default this parameter is set to none REPORT INSTANCE DATACENTER By default this test reports the availability of only those availability zones that contain one more instances Accordingly this flag is set to true by default If you want the test to report metrics for all availability zones regardless of whether not they host instances set this flag to false 42 MONITORING THE AWS EC2 REGION 10 11 12 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a val
20. test Measurement Measurement Measurement Interpretation made by the Unit P test Availability Number The value 0 indicates that the availability zone is Indicates whether not Not Available and the value 100 indicates that it is this availability zone is Available currently available If an availability zone fails then all server instances operating within that zone will also be rendered unavailable If you host all your Amazon EC2 instances in a single location that is affected by such a failure your instances will be unavailable thereby bringing your entire application to a halt On the other hand if you have instances distributed across many Availability Zones and one of the instances fails you can design your application so the instances in the remaining Availability Zones handle any requests 43 MONITORING THE AWS EC2 REGION 3 1 2 EC2 Regions Test Amazon EC2 provides the ability to place instances in multiple locations Amazon EC2 locations are composed of Availability Zones and Regions Regions are dispersed and located in separate geographic areas US EU etc Each Region is completely independent By launching instances in separate Regions you can design your application to be closer to specific customers or to meet legal or other requirements Amazon EC2 Ee s X Figure 3 3 Regions and Availability zones If a region is unavailable then users to that region will not be able to
21. their server instances are inaccessible you may want to know whether it is because of the non availability of the availability zone within which the instances have been launched This test auto discovers the 5 MONITORING THE AWS EC2 CLOUD regions and availability zones on the Amazon EC2 Cloud and reports the availability of each zone Purpose Auto discovers the regions and availability zones on the Amazon EC2 Cloud and reports the availability of each zone Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a correspondin
22. through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG suite embeds an optional detailed diagnostic capability With this capability the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a
23. very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 71 MONITORING THE AWS EC2 REGION 10 11 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed
24. AMI might contain all the software to act as a web server e g Linux Apache and your web site or it might contain all the software to act as a Hadoop node e g Linux Hadoop and a custom application After an AMI is launched the resulting running system is called an instance All instances based on the same AMI start out identical and any information on them is lost when the instances are terminated or fail Users with valid AWS user accounts can sign into an EC2 region to view and use available instances or purchase and launch new ones With the help of this test you can determine the total number of instances that are currently available for the configured AWS user account in the monitored region the number of instances that were newly purchased terminated and the count of powered off instances Purpose Helps determine the total number of instances that are currently available for the configured AWS user account in the monitored region the number of instances that were newly purchased terminated and the count of powered off instances 56 MONITORING THE AWS EC2 REGION Target of the Amazon EC2 Region test Agent A remote agent deploying the test Configurable 1 TEST PERIOD How often should the test be executed elutes fer 2 HOST The host for which the test is being configured 3 AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with
25. C2 Region T The tests mapped to the AWS EC2 Region Status layer sessssesseseseeeeeneneenne nennen nnne theres entree nn enne nete tenen enne nennen Regions and Availability ZONES sciiicet ee RERO YER EHEEN VERSER ERE HR EXT HEERE HR FEE EXE ERE EYE RE TENPE EEEN TEA SERERE RR ERR EE The tests mapped to the AWS EC2 Region Instance Status layer The detailed diagnosis of the Total instances measure cecceceesceseeseeeesceeceeseeseeseesecsecsseseeseesecsecseceeeaeeseeseesecaeeeeeaeeaeeaeeaeeeeeneeeaeeaeeas The detailed diagnosis of the Instances powered on measure enne enne treten ennt tn ennt ns entente nnn eetet The detailed diagnosis of the Instances powered off measure The tests mapped to the AWS EC2 Region Instance Details layer ade The detailed diagnosis of the Has VM been rebooted measure sse netten enne trennen tnn nen nennen 73 INTRODUCTION Introduction Amazon Elastic Compute Cloud Amazon EC2 is a web service that provides resizeable computing capacity literally server instances in Amazon s data centers that you use to build and host your software systems You can get access to the infrastructure resources that EC2 provides by using APIs or web tools and utilities With EC2 you use and pay for only the capacity that you need This eliminates the need to make large and expensive hardware purchases reduces the need to forecast traffic and enables you to automatically scale your
26. ING THE AWS EC2 CLOUD test Availability Indicates whether not this availability zone in this region is currently available Number The value 0 indicates that the availability zone is Not Available and the value 100 indicates that it is Available If an availability zone fails then all server instances operating within that zone will also be rendered unavailable If you host all your Amazon EC2 instances in a single location that is affected by such a failure your instances will be unavailable thereby bringing your entire application to a halt On the other hand if you have instances distributed across many Availability Zones and one of the instances fails you can design your application so the instances in the remaining Availability Zones handle any requests 2 2 1 AWS EC2 Server Logins Test This test attempts to connect to the default region in the cloud in the process the test reports whether the configured AWS user account is able to access the cloud based infrastructure or not and if so how quickly the connection with the infrastructure was established If a user is denied access to a server instance on a clod or if a user experiences a significant delay in connecting to his her instances you can use this test to validate the user credentials and to figure out whether any connectivity issues exist Purpose Attempts to connect to the default region in the cloud in the process
27. KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web
28. LOUD Some instance types are designed for standard applications whereas others are designed for CPU intensive applications or memory intensive applications etc The different instance types offered by the AWS EC2 cloud are as follows 26 MONITORING THE AWS EC2 CLOUD yom Cm Mem tois 1 EC2 Compute Unit 1 virtual core with 1 EC2 Compute Unit 4 EC2 Compute Units 2 virtual cores with 2 EC2 Compute Units each 8 EC2 Compute Units 4 virtual cores with 2 EC2 Compute Units each 160 GB instance storage 150 GB plus 10 GB root partition 850 GB instance storage 2 x 420 GB plus 10 GB root partition storage 4 x 420 GB plus 10 GB root partition High CPU Medium High CPU Extra Large High Memory Extra Large High Memory Double Extra Large High Memory Quadruple Extra Large Up to 2 EC2 Compute Units for short periodic bursts 5 EC2 Compute Units 2 virtual cores with 2 5 EC2 Compute Units each 20 EC2 Compute Units 8 virtual coms with 2 5 EC2 Compute Units each 6 5 EC2 Compute Units 2 virtual coms with 3 25 EC2 Compute Units each 13 EC2 Compute Units 4 virtual coms with 3 25 EC2 Compute Units each 26 EC2 Compute Units 8 virtual cores with 3 25 EC2 Compute Units each EBS volumes for storage 350 GB instance storage 340 GB plus 10 GB root partition 1690 GB instance storage 4 x420 GB plus 10 GB root partition 420 GB instance s
29. Monitoring the AWS EC2 Cloud eG Enterprise V6 Restricted Rights Legend The information contained in this document is confidential and subject to change without notice No part of this document may be reproduced or disclosed to others without the prior permission of eG Innovations Inc eG Innovations Inc makes no warranty of any kind with regard to the software and documentation including but not limited to the implied warranties of merchantability and fitness for a particular purpose Trademarks Microsoft Windows Windows NT Windows 2003 and Windows 2000 are either registered trademarks or trademarks of Microsoft Corporation in United States and or other countries The names of actual companies and products mentioned herein may be the trademarks of their respective owners Copyright 2015 eG Innovations Inc All rights reserved Table of Contents INTRODUCTION 33 1 in oci Eo he n d nda indo vU iu M nA MONITORING THE AWS EC2 CLOUD 2 1 The AWS Cloud Statis Layer 2 ioter trier rb ertet e e ri eee beet e FU eade Fee PESE E rd RE FERE REENE ERE OTE iE 5 2 22 JAWS EC2 Availability Zones Testseite aisis eria ri olei te E T lere ple aeri a oboe Ce e EEAS 5 2 2 1 AWS EC2 Server Logins Test ern bee e ae ERE p REEL EROS 9 2 2 2 AAW S EC2 Web Access T68t eres poc ER IHRER aiscniuecaszsdeas saves EEFEXAE EM RENS EUIS REX RE DREAM EET EE RXE FAEERE NER EOS 12 2 2 3 AWS EC2 Regions Test ect teer e rt re PEU OR Fei aini FI
30. PU excessively this could be percentage of because of l one more resource intensive allocated CPU processes executing on that instance consumed by this Compare the value of this measure across instance instances to identify the CPU intensive instances Incoming network KB Sec Compare the values of these measures across traffic instances to quickly identify the instance that is utilizing the network bandwidth excessively Indicates the rate of incoming network traffic i e the rate at which the bytes are received by all the network interfaces connected to this instance Outgoing network KB Sec traffic Indicates the volume of outgoing network traffic i e the rate at which the bytes are transferred from all the network interfaces connected to this instance 34 MONITORING THE AWS EC2 CLOUD These measures are good indicators of the level of disk I O activity on an instance By comparing the values of these measures across instances you can accurately determine which instance is performing I O intensive operations Disk reads KB Sec Indicates the rate at which data is read from the disks of this instance Disk writes KB Sec Indicates the rate at which data is written to the disks of this instance Disk read Operations Sec operations Indicates the rate at which disk read operations are performed on this instance Disk write Operations Sec
31. This measure quantifies the must be very small of the order of a few time for establishing a TCP milliseconds connection to the configured URL Server response time Secs While the total response time may depend on This measure indicates the several factors the server response time is time period between when the typically a very good indicator of a server connection was established bottleneck e g because all the available and when the test sent back a server threads or processes are in use HTTP response header to the client Response code Number A value between 200 and 300 indicates a The response code returned by good response A 4xx value indicates a the test for the simulated problem with the requested content eg request page not found A 5xx value indicates a server error Content length Kbytes Typically the content length returned by the The size of the content test for a specific URL should be the same returned by the test across time Any change in this metric may indicate the need for further investigation Content validity Percent A value of 100 indicates that the content This measure validates returned by the test is valid A value of 0 whether the test was indicates that the content may not be valid successful in executing the This capability for content validation is request made to it especially important for multi tier web applications For example a user may not be able to login to the web s
32. access e Some tests require the AWS CloudWatch service to be enabled This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics For enabling this service you need to pay CloudWatch fees Refer to the AWS web site for the fee details The sections that follow will discuss each layer of Figure 3 1 elaborately 40 MONITORING THE AWS EC2 REGION 3 1 The AWS EC2 Region Status Layer Using the tests mapped to this layer you can promptly detect the non availability of a target region and the availability zones in that region and connection bottlenecks experienced while connecting to the cloud or its components AWS EC2 Region Status seach JO We EC2 Availability Zones E ap southeast 1a EC2 Regions Fd AWS EC2 Web Access E HomePage Figure 3 2 The tests mapped to the AWS EC2 Region Status layer 3 1 1 EC2 Availability Zones Test Amazon has data centers in different areas of the world e g North America Europe Asia etc Correspondingly EC2 is available to use in different Regions Each Region contains multiple distinct locations called Availability Zones illustrated in the following diagram Each Availability Zone is engineered to be isolated from failures in other Availability zones and to provide inexpensive low latency network connectivity to other zones in the same Region By launching instances in separ
33. ailability of the cloud over the web and the time it took for the agent to access the cloud over the web This way issues in web based access to the cloud come to light Purpose Emulates a user accessing a web page by default the login page on the cloud via HTTP S and reports whether that page is accessible or not In the process the test indicates the availability of the cloud over the web and the time it took for the agent to access the cloud over the web Target An AWS EC2 cloud Agent A remote agent deploying this test Configurable 1 TEST PERIOD How often should the test be executed parameters for gt URL The web page being accessed While multiple URLs separated by commas can be 46 MONITORING THE AWS EC2 REGION this test 10 11 12 13 provided each URL should be of the format URL name URL value URL name is a unique name assigned to the URL and the URL value is the value of the URL By default the url parameter is set to HomePage http aws amazon com ec2 where HomePage is the URL name and Attp aws amazon cor ec2 is the URL value You can modify this default setting to configure any URL of your choice eg the URL of the login page to your cloud based infrastructure HOST The host for which the test is to be configured PORT The port to which the specified HOST listens COOKIEFILE Whether any cookies being returned by the web server need to be
34. all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for each instance available for the configured AWS user account test Measurements Measurement made by the Measurement Unit Interpretation test Avg network delay Secs An increase in network latency co
35. ata from metrics For enabling this service you need to pay CloudWatch fees Refer to the AWS web site for the fee details The sections that will follow discuss each of the layers of Figure 2 1 in great detail 2 1 The AWS Cloud Status Layer Using the tests mapped to this layer you can promptly detect the non availability of the cloud inaccessibility of regions and availability zones on the cloud and connection bottlenecks experienced while connecting to the cloud or its components AWS Cloud Status seac V all Be AWS EC2 Datacenter Status Li E ap southeast 1 ap southeast 1a Wi us east 1 us east 1d AWS EC2 Server Login Status Aw AWS EC2 Web Access BN HomePage e AWS EC2 Region Status Li E ap southeast 1 W us east 1 Figure 2 2 The test associated with the Hardware layer 2 2 AWS EC2 Availability Zones Test Amazon has data centers in different areas of the world e g North America Europe Asia etc Correspondingly EC2 is available to use in different Regions Each Region contains multiple distinct locations called Availability Zones illustrated in the following diagram Each Availability Zone is engineered to be isolated from failures in other Availability zones and to provide inexpensive low latency network connectivity to other zones in the same Region By launching instances in separate Availability Zones you can protect your applications from the failure of a single location If users complaint that
36. ate Availability Zones you can protect your applications from the failure of a single location If users complaint that their server instances are inaccessible you may want to know whether it is because of the non availability of the availability zone within which the instances have been launched This test auto discovers the availability zones configured within the monitored EC2 region and reports the availability of each zone Purpose Auto discovers the availability zones configured within the monitored EC2 region and reports the availability of each zone Target of the Amazon EC2 Region test Agent A remote agent deploying the test 41 MONITORING THE AWS EC2 REGION Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS acco
37. ation with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for each instance launched by the configured AWS user account in the test monitored region Measurement Measurement Measurement Interpretation made by the Unit test CPU utilization Percent A high value for this measur
38. blish a TCP connection to process is not operating correctly In some this URL cases of extreme overload the failure to establish a TCP connection may be a transient condition As the load subsides the web page may start functioning properly again Tcp connect time Secs Typically the TCP connection establishment This measure quantifies the must be very small of the order of a few time for establishing a TCP mulliseconds connection to the configured URL Server response time Secs While the total response time may depend on This measure indicates the several factors the server response time is time period between when the typically a very good indicator of a server connection was established bottleneck e g because all the available and when the test sent back a server threads or processes are in use HTTP response header to the client Response code Number A value between 200 and 300 indicates a The response code returned by good response A 4xx value indicates a the test for the simulated problem with the requested content eg request page not found A 5xx value indicates a server error Content length Kbytes Typically the content length returned by the The size of the content returned by the test test for a specific URL should be the same across time Any change in this metric may indicate the need for further investigation 14 MONITORING THE AWS EC2 CLOUD Content validity Thi
39. ccount e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes
40. ce was powered on successfully then powered on the value of this measure will be Yes If not then sucessful the value of this measure will be Vo Indicates whether not the configured instance was powered on successfully The numeric values that correspond to the measure values mentioned above are as follows Measure value Numeric value Yes 100 No 0 Note By default this measure reports one of the Measure values listed in the table above The graph of this measure however represents the success failure of a power on operation using the numeric equivalents 0 and 100 only Was instance powered off sucessful Indicates whether not the configured instance was powered off successfully If the instance was powered off successfully then the value of this measure will be Yes If not then the value of this measure will be Wo The numeric values that correspond to the measure values mentioned above are as follows Measure value Numeric value Yes 100 No 0 Note By default this measure reports one of the Measure values listed in the table above The graph of this measure however represents the success failure of a power off operation using the numeric equivalents 0 and 100 only 52 MONITORING THE AWS EC2 REGION Time taken for Secs Ideally the value of these measures should be instance to be low A sudden increase in the val
41. ched removed recently If so which ones are these e What type of instances are resource intensive e Isany particular instance consuming too much CPU e Is the network traffic to from any instance unusually high e Is the disk I O of instances optimal e Was any instance rebooted recently If so which one is it To enable the eG agent to collect these useful metrics the following pre requisites need to be fulfilled e The eG agent should be deployed on a remote Windows host in the environment e The eGurkhaAgent service of the remote agent should run using domain administrator privileges To know how to set this up refer to the eG User Manual e Each test executed by the remote agent uses the AWS API to collect the required metrics To enable the tests to access the AWS API you need to configure the tests with the access key and password of a user with a valid AWS user account To obtain this access key do the following o Sign up for a new AWS account from the http aws amazon com ec2 page o Provide the details of the user for whom you wish to create the AWS account o Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 o Once the payment is made the user will be automatically signed in to the AWS account Note The eG agent reports metrics for only availability zones and instances in a region that the configured AWS user account is allowed to
42. consumed by all Instances instances of this type Compare the value of this measure across types to identify the types of instances that are CPU intensive Incoming network KB Sec Compare the values of these measures across traffic instance types to quickly identify the types of instances that are utilizing the network bandwidth Indicates the rate of incoming network traffic i e the rate at which the bytes are received by all the network interfaces connected to all the instances of this instance type excessively 30 MONITORING THE AWS EC2 CLOUD Outgoing network KB Sec traffic Indicates the volume of outgoing network traffic i e the rate at which the bytes are transferred from all the network interfaces connected to all the instances of a particular instance type Disk reads KB Sec These measures are good indicators of the level Indicates the rate at of disk I O activity on an instance type By which data is read comparing the values of these measures across from the disks of all types you can accurately determine the type of instances of this type instances that is performing I O intensive operations Disk writes KB Sec Indicates the rate at which data is written to the disks of all instances of this type Disk read Operations Sec These measures are good indicators of the level operations of disk I O activity on an instance type By Indicate
43. d PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default 10 PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for the AWS EC2 region being monitored test ESM IUS Measurement Mea menani Interpretation made by the Unit p test Availability Number The value O indicates that the region is Not Available and the value 100 indicates that it is Indicates whether not the region is currently Available available Response time Secs A low value is typically desired for this measure A high value or a consistent increase in this value could be indicative of connection bottlenecks Indicates the time taken to connect to the region 3 1 3 AWS EC2 Web Access Test This test emulates a user accessing a web page on the cloud via HTTP S and reports whether that page is accessible or not In the process the test indicates the av
44. d flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 17 MONITORING THE AWS EC2 CLOUD 8 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none in
45. dicating that the eG agent is not configured to communicate via a proxy by default 9 PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default 10 PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for each region of the AWS EC2 Cloud being monitored test Measurements Measurement t Interpretation made by the Measurement Unit terpretatio test Availability Number The value O indicates that the region is Not Indicates whether not Available and the value 100 indicates that it is this this region is Available currently available Response time Secs A low value is typically desired for this measure A high value or a consistent increase in this value could be indicative of connection bottlenecks Indicates the time taken to connect to this region Compare the value of this measure across regions to know which region takes the
46. e Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the following conditions are fulfilled e The eG manager license should allow the detailed diagnosis capability e Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0 Outputs of the One set of results for each instance launched by the configured AWS user account test Measurements Measurement Measurement Interpretation made by the Unit p test Has the instance Boolean If this measure shows 1 it means that the been rebooted instance was rebooted during the last Indicates whether this instance has been rebooted during the last period or not measurement period By checking the time periods when this metric changes from 0 to 1 an administrator can determine the times when this measurement instance was rebooted 37 MONITORING THE AWS EC2 CLOUD Uptime of the instance during the last measure period Indicates the time period that the instance has been up since the last time this test ran Secs If the instance has not been rebooted during the last measurement period and the agent has been running continuously this value will be equal to the measurement period If the instance was rebooted during the last measurement period this value will be less than the measurement period of the test For example if the meas
47. e access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data fr
48. e indicates that an Indicates the instance is utilizing CPU excessively this could be percentage of because of l one more resource intensive allocated CPU processes executing on that instance consumed by this Compare the value of this measure across instance instances to identify the CPU intensive instances Incoming network KB Sec Compare the values of these measures across traffic instances to quickly identify the instance that is utilizing the network bandwidth excessively Indicates the rate of incoming network traffic i e the rate at which the bytes are received by all the network interfaces connected to this instance Outgoing network KB Sec traffic Indicates the volume of outgoing network traffic i e the rate at which the bytes are transferred from all the network interfaces connected to this instance 69 MONITORING THE AWS EC2 REGION These measures are good indicators of the level of disk I O activity on an instance By comparing the values of these measures across instances you can accurately determine which instance is performing I O intensive operations Disk reads KB Sec Indicates the rate at which data is read from the disks of this instance Disk writes KB Sec Indicates the rate at which data is written to the disks of this instance Disk read Operations Sec operations Indicates the rate at which disk read operations are
49. e rate at comparing the values of these measures across which disk read types you can accurately determine the type of operations were instances that is performing I O intensive performed on the disks operations of all instances of this type Disk write Operations Sec operations Indicates the rate at which disk write operations were performed performed on the disks of all instances of this type 3 3 2 Tracking the CPU usage disk and network I O of every instance launched by a configured AWS user account in a region will provide administrators with valuable insights into how well the instances are utilizing the allocated resources The EC2 Instance Resources test does just that This test auto discovers the instances available for the configured AWS user account in a region and reports the resource usage of each instance so that administrators can quickly compare the usage metrics across instances and pinpoint which instance is resource hungry EC2 Instance Resources Test 66 MONITORING THE AWS EC2 REGION Purpose Auto discovers the instances available in the monitored region for the configured AWS user account and reports the resource usage of each instance so that administrators can quickly compare the usage metrics across instances and pinpoint which instance is resource hungry Target of the Amazon EC2 Region test Agent A remote agent deploying the test 67
50. e the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to rue On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 36 MONITORING THE AWS EC2 CLOUD 10 11 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG
51. e to access the configured URL or not several factors such as the web server process es hosting the configured web page being down the web server being misconfigured a network failure etc Temporary unavailability may also occur if the web server is overloaded Availability is determined based on the response code returned by the test A response code between 200 to 300 indicates that the configured web page is available Total response time Secs Response time being high denotes a problem This measurement indicates the time taken by the test to access this URL Poor response times may be due to an overload If the URL accessed involves the generation of dynamic content backend problems e g an overload at the application server or a database failure can also result in an increase in response time Tcp connection availability Percent Failure to establish a TCP connection may This measure indicates whether the test managed to establish a TCP connection to this URL imply that either the web server process hosting the web page is not up or that the process is not operating correctly In some cases of extreme overload the failure to establish a TCP connection may be a transient condition As the load subsides the web page may start functioning properly again 48 MONITORING THE AWS EC2 REGION Tcp connect time Secs Typically the TCP connection establishment
52. e total if enabled shows the details of all the instances that were newly purchased and launched by the configured AWS user account number of instances that were newly purchased by the configured AWS user account during the last measurement period Removed instances Number The detailed diagnosis capability of this measure if enabled shows the details of all the instances that were newly terminated removed by the configured AWS user account Indicates the total number of instances that were newly terminated by the configured AWS user account during the last measurement period The detailed diagnosis capability of the 7ota instances measure if enabled shows the details of all the instances available for use for the configured AWS account in the monitored region regardless of the current state of the instances Component myec2 east region Measured By ec2remote Test EC2 Instances Measurement Total instances v Timeline 1 hour v From Details of Instances in AWS EC2 Time IP Address l Type Monitoring 09 07 11 02 32 27 ami 93ec93ci 122 248 198 156 wi ve mi small ap southeast ia mi small ap southeast 1a mi small ap southeast 1a mi small ap southeast 1a 93c5 122 v mi small ap southeast 1a c93 c93c6 N A v mi small ap southeast ia enabled i bOc3efe8 ami S3ecS3c7 WA va mi small ap southeast 18 enabled Figure 3 5 The detailed diagnosis of
53. ecret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account INSTANCE ID Specify the ID of the instance that is to be powered on off by this test By default this test is not designed to report any metrics for any instance this is why the instance id is set to none by default Note that this test will not run until a valid instance id and instance type are provided INSTANCE TYPE Specify the type of the configured instance Since this test is not designed to report any metrics for any instance by default this parameter is set to none by default Note that this test will report metrics only if a valid instance id and instance type are provided FREQUENCY Indicate how frequently in seconds the test needs to power an instance on and off By default the test period and frequency of this test will be the same both will be set to 24 hrs i e 86400 seconds by default However some administrators may not want their critical instances to be powered on and off at the same frequency at which the test runs In such a case you can define a separate power on off frequency for the test using the frequency parameter PROXYHOST and PROXY PORT In some environment
54. ed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the test One set of results for each instance available for the configured AWS user account in the monitored region 55 MONITORING THE AWS EC2 REGION Measurements Measurement made by the Measurement Unit Interpretation test Avg network delay Secs An
55. es the total number of instances that were newly purchased by the configured AWS user account during the last measurement period Removed instances Number The detailed diagnosis capability of this measure total if enabled shows the details of all the instances that were newly terminated removed by the configured AWS user account Indicates the number of instances that were newly terminated by the configured AWS user account during the last measurement period 2 4 The AWS Cloud Instance Details Layer The tests mapped to this layer auto discover the server instances that are available for the configured AWS user account on the cloud and reports the uptime and the resource usage of the individual instances 7 AD A F4 AWS Cloud Instance Details Search O e AWS EC2 Aggregated Resource Usage W ap southeast 1 mi small W us east 1 mi small e AWS EC2 Instance Resources B ap southeast 1 karthika i b0c3efe2 W us east 1 karthika i 07dbe56d We AWS EC2 Instance Uptime W ap southeast 1 karthika i bOc3efe2 W us east 1 karthika i 07dbe56d Figure 2 5 The tests mapped to the AWS Cloud VM Details layer 2 4 1 AWS EC2 Aggregated Resource Usage Test When users launch an instance using the AWS management console they need to specify the instance type An instance type is a specification that defines the memory CPU storage capacity and hourly cost for an instance 25 MONITORING THE AWS EC2 C
56. f instance has been up the test For example if the measurement period since the last time this is 300 secs and if the instance was rebooted 120 test ran secs back this metric will report a value of 120 seconds The accuracy of this metric is dependent on the measurement period the smaller the measurement period greater the accuracy Total uptime of the Mins Administrators may wish to be alerted if an instance instance has been running without a reboot for a Indicates the total time very long period Setting a threshold for this that this intance has metric allows administrators to determine such been up since its last conditions reboot The detailed diagnosis of the Has VM been rebooted measure reveals when the instance was last shutdown when it was rebooted how long the shutdown lasted and whether the instance was shutdown as part of a routine maintenance exercise 192 168 8 164 karthikati bOc3efe2 aws ap southeast 1 Measured By Description Component Test EC2 VM Uptime Measurement Has VM been rebooted From E i To E Min Timeline ihour LJ Last rebooted details ShutDownDate RebootDate Jul 15 2011 17 37 54 ShutDownDuration Mins Time Jul 15 2011 17 43 28 Jun 22 2011 16 07 42 33210 21 Figure 3 9 The detailed diagnosis of the Has VM been rebooted measure 73 CONCLUSION Conclusion Thi
57. for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics
58. g secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Reso
59. ge 1 x840 GB plus 10 GB root partition 1690 GB instance 64 bit storage 2 x840 GB plus 10 GB root partition Cluster GPU 33 5 EC2 Compute 23 GB Units 2 x Intal Xeon X5570 quad core Nehalem architecture 33 5 EC2 Compute 22 GB Units 2 x Intal Xeon see note X5570 quad core after this Nehalem architecture plus 2 NVIDIA Tesla M2050 Fermi GPUs 1690 GB instance 64 bit 64 bit storage 2 x 840 GB plus 10 GB root partition 1690 GB instance 64 bit storage 2 x 840 GB plus 10 GB root parlition MONITORING THE AWS EC2 REGION By closely monitoring the CPU usage and the network and disk I O of each instance type and comparing these metrics across instance types you can quickly isolate resource intensive types Once again the test will report metrics for only those types of instances that were launched by the AWS user account configured for the test in the monitored region Purpose Closely monitors the CPU usage and the network and disk I O of each instance type and enables usage comparison across instance types so as to quickly isolate resource intensive types Target of the Amazon EC2 Region test Agent A remote agent deploying the test 63 MONITORING THE AWS EC2 REGION Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS
60. ge metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 54 MONITORING THE AWS EC2 REGION 10 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be rout
61. he eG manager license should allow the detailed diagnosis capability e Both the bad and normal frequencies configured for the detailed diagnosis measures should not be 0 Outputs of the One set of results for the AWS EC2 Cloud being monitored test CEST Measurement Measurement Interpretation made by the Unit P test Total instances Number The detailed diagnosis capability of this measure Indicates the total if enabled shows the details of all the instances number of instances available for use for the configured AWS account currently available for regardless of the current state of the instances the configured AWS user account Instances powered Number The detailed diagnosis capability of this measure on if enabled shows the details of all the powered Indicates the total on instances available for use for the configured number of instances AWS account that are currently powered on 24 MONITORING THE AWS EC2 CLOUD Instances powered Number The detailed diagnosis capability of this measure off if enabled shows the details of all the powered Indicates the total off instances available for the configured AWS number of instances account that are currently powered off Added instances Number The detailed diagnosis capability of this measure if enabled shows the details of all the instances that were newly purchased and launched by the configured AWS user account Indicat
62. icing on performance You need performance management FOR the cloud eG Enterprise is a unique solution that can provide you performance management FROM the cloud OF the cloud and FOR the cloud INTRODUCTION Private Public Cloud Cloud Services Cloud service providers can optimize their cloud services and ensure Customer satisfaction They can offer performance Service management of cloud hosted apps Manager en a EES Cloud as a value added service Provider Figure 1 1 How eG monitors the cloud To deliver performance management FOR the AWS EC2 cloud in particular the solution offers two specialized monitoring models the AWS EC2 Cloud model and the AWS EC2 Region model The AWS EC2 Cloud monitoring model provides you with proactive updates on the overall health and status of the cloud and points you to unavailable regions availability zones and resource hungry instances in the cloud To zoom into the health of specific regions and the instances operating within those regions use the AWS EC2 Region model This document engages in detailed discussions on both the models MONITORING THE AWS EC2 CLOUD Monitoring the AWS EC2 Cloud Figure 2 1 depicts the AWS EC2 Cloud monitoring model that eG Enterprise offers out of the box for monitoring the Amazon EC2 cloud v AWS Cloud Instance Details v AWS Cloud Instance Status AWS Cloud Status Figure 2 1 Layer model of the AWS EC2 Cloud Each layer of thi
63. id proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG system embeds an optional detailed diagnostic capability With this capability the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a particular server choose the On option against DETAILED DIAGNOSIS To disable the capability click on the Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the following conditions are fulfilled e The eG manager license should allow the detailed diagnosis capability e Both the bad and normal frequencies configured for the detailed diagnosis measures should not be 0 Outputs of the One set of results for each availability zone in the AWS EC2 Region being monitored
64. igurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to re
65. ils Ch AWS EC2 Region Instance Status r1 AWS EC2 Region Status Figure 3 1 The layer model of the AWS EC2 Region In addition using a single eG agent installed on a remote Windows host in the environment the model auto discovers the IP address and the operating system of the instances launched on the cloud periodically checks the powered on status of each of the instances continously assesses how each instance is utilizing the allocated resources and thus promptly alerts you to unavailable and resource hungry instances As the solution also automatically determines what applications have been deployed on the instances whenever one of these applications experience slowdowns administrators can use the eG solution to instantly and accurately diagnose the root cause of the slowdown is it owing to the corresponding instance being unavailable or the application being resource hungry Using the metrics so reported administrators can ascertain the following e Is web based HTTP HTTPS access to the region available e Does it take an unreasonably long time to establish contact with the region e How many availability zones exist in the monitored region What are they e Is any availability zone currently unavailable If so which one is it 39 MONITORING THE AWS EC2 REGION e Are all instances launched in the region accessible over the network e Are any instances powered off currently e Were any instances laun
66. increase in network latency could result Indicates the average delay from misconfiguration of the router s along between transmission of the path network congestion packets to this instance retransmissions at the network etc and receipt of the response to the packet at the source Min network delay Secs A significant increase in the minimum round The irritum time trip time is often a sure sign of network between transmission of a congestion packet and receipt of the response back Packet loss Percent Packet loss is often caused by network buffer Indicates the percentage of overflows at a network router or by packet packets lost during corruptions over the network The detailed wansmiesiam fromcsource to diagnosis for this measure provides a listing target and back of routers that are on the path from the external agent to target server and the delays on each hop This information can be used to diagnose the hop s that could be causing excessive packet loss delays Network availability of Percent A value of 100 indicates that the instance is instance accessible over the network The value 0 Indicates whether the indicates that the instance is inaccessible network connection to this Typically the value 100 corresponds to a instance is available or not Packet loss of 0 3 2 3 EC2 Instances Test An Amazon Machine Image AMI contains all information necessary to boot instances of your software For example an
67. ions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for each instance launched by the configured AWS user account test Measurements Measurement Measurement Interpretation made by the Unit test CPU utilization Percent A high value for this measure indicates that an indicates the instance is utilizing C
68. ite but the server may reply back with a valid HTML page where in the error message say Invalid Login is reported In this case the availability will be 100 since we got a valid HTML response If the test is configured such that the content parameter should exclude the string Invalid Login in the above scenario content validity would have a value 0 3 2 The AWS EC2 Region Instance Status Layer To determine issues in accessibility server instances launched in a region and to detect the current state of each instance use the tests mapped to this layer 49 MONITORING THE AWS EC2 REGION AWS EC2 Region Instance Status sesc sD v all EC2 Instance Deployment EC2 Instances Figure 3 4 The tests mapped to the AWS EC2 Region Instance Status layer 3 2 1 EC2 Instance Deployment Test This test powers a specified VM on and off at configured intervals In the process the test verifies the success failure of the corresponding operation i e power on off and also reports the time taken by that instance to power on and off Failed attempts to power on and significant delays in powering on are thus brought to light This test is disabled by default To enable the test go to the ENABLE DISABLE TESTS page using the menu sequence Agents Tests Enable Disable pick the desired Component type set Performance as the Test type choose the test from the DISABLED TESTS list and click on the
69. l be displayed corresponding to each listed URLname a Username text box and a Password text box will be made available These parameters will take either of the following values a a valid Username and Password for every configured URLname b nonein both the Username and Password text boxes of all configured URLnames the default setting if no user authorization is required Where NTLM Integrated Windows authentication is supported valid CREDENTIALS are mandatory In other words a none specification will not be supported in such cases Therefore in this case against each configured URLname you will have to provide a valid Username in the format domainname username followed by a valid Password Please be sure to check if your web site requires HTTP authentication while configuring this parameter HTTP authentication typically involves a separate pop up window when you try to access the page Many sites use HTTP POST for obtaining the user name and password and validating the user login In such cases the username and password have to be provided as part of the POST information and NOT as part of the CREDENTIALS specification for the this test PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments 47 MONITORING THE AWS EC2 REGION 14 15 16 you should make sure that the eG agent connects to the c
70. lity the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a particular server choose the On option against DETAILED DIAGNOSIS To disable the capability click on the Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the following conditions are fulfilled e The eG manager license should allow the detailed diagnosis capability e Both the bad and normal frequencies configured for the detailed diagnosis measures should not be 0 Outputs of the One set of results for the AWS EC2 Cloud being monitored test Measurements Measurement Measurement Interpretation made by the Unit P test Default region Percent The value O0 indicates that the region is not availability accessible and the value 100 indicates that it is Indicates whether or accessible If the default region is inaccessible it not the the test is able could be owing to any one of the following to access the default reasons region on the cloud q The cloud is unavailable using the configured AWS user account r The configured AWS account does not have the access rights to the default region s The test has been configured with incorrect login credentials 11 MONITORING THE AWS EC2 CLOUD Response time Secs A low value is desired for this
71. loud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none TIMEOUT Here specify the maximum duration in seconds for which the test will wait for a response from the server The default TIMEOUT period is 30 seconds Outputs of the One set of outputs for every URL being monitored test 1 CTT IUS Measurement Measurement Interpretation of the test Unit P Availability Percent Availability failures could be caused by This measurement indicates whether the test was abl
72. measure A high Indicates the time value or a consistent increase in this value could taken by the test to indicate connection bottlenecks establish a connection with the default region on the cloud 2 2 2 AWS EC2 Web Access Test This test emulates a user accessing a web page on the cloud via HTTP S and reports whether that page is accessible or not In the process the test indicates the availability of the cloud over the web and the time it took for the agent to access the cloud over the web This way issues in web based access to the cloud come to light Purpose Emulates a user accessing a web page by default the login page on the cloud via HTTP S and reports whether that page is accessible or not In the process the test indicates the availability of the cloud over the web and the time it took for the agent to access the cloud over the web Target An AWS EC2 cloud Agent A remote agent deploying this test Configurable parameters for this test 1 TEST PERIOD How often should the test be executed 2 URL The web page being accessed While multiple URLs separated by commas can be provided each URL should be of the format URL name URL value URL name is a unique name assigned to the URL and the URL value is the value of the URL By default the url parameter is set to HomePage http aws amazon com ec2 where HomePage is the URL name and ttp aws amazon com ec2 is the URL value Yo
73. med by all Instances instances of this type Compare the value of this measure across types to identify the types of instances that are CPU intensive Incoming network KB Sec Compare the values of these measures across traffic instance types to quickly identify the types of instances that are utilizing the network bandwidth Indicates the rate of incoming network traffic i e the rate at which the bytes are received by all the network interfaces connected to all the instances of this instance type 65 excessively MONITORING THE AWS EC2 REGION Outgoing network KB Sec traffic Indicates the volume of outgoing network traffic i e the rate at which the bytes are transferred from all the network interfaces connected to all the instances of a particular instance type Disk reads KB Sec These measures are good indicators of the level Indicates the rate at of disk I O activity on an instance type By which data is read comparing the values of these measures across from the disks of all types you can accurately determine the type of instances of this type instances that is performing I O intensive operations Disk writes KB Sec Indicates the rate at which data is written to the disks of all instances of this type Disk read Operations Sec These measures are good indicators of the level operations of disk I O activity on an instance type By Indicates th
74. o the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the AWS EC2 VM Resource Usage and AWS EC2 VM Aggregate Resource usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to a se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests In the EXCLUDE INSTANCE text box provide a comma separated list of instance names or instance name patterns that you do not wish to monitor For example b0c3e 7dbe56d By default this parameter is set to none
75. om metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 23 MONITORING THE AWS EC2 CLOUD 10 11 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server li
76. on can include wild card patterns For example an Inc instruction can be Inc Home page An Inc and an Exc instruction can be provided in quick succession in the following format Jnc Home Page Exc home CREDENTIALS The HttpTest supports HTTP authentication The CREDENTIALS parameter is to be set if a specific user name password has to be specified to login to a page Against this parameter the URLname of every configured url will be displayed corresponding to each listed URLname a Username text box and a Password text box will be made available These parameters will take either of the following values a a valid Username and Password for every configured URLname b nonein both the Username and Password text boxes of all configured URLnames the default setting if no user authorization is required Where NTLM Integrated Windows authentication is supported valid CREDENTIALS are mandatory In other words a none specification will not be supported in such cases Therefore in this case against each configured URLname you will have to provide a valid Username in the format domainname username followed by a valid Password Please be sure to check if your web site requires HTTP authentication while configuring this parameter HTTP authentication typically involves a separate pop up window when you try to access the page Many sites use HTTP POST for obtaining the user name and password and validating the user login In such case
77. onfigured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch ser
78. or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 45 MONITORING THE AWS EC2 REGION 8 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default 9 PROXY USERNAME an
79. or which the test will wait for a response from the server The default TIMEOUT period is 30 seconds Outputs ofthe One set of outputs for every URL being monitored test Measurements Measurement Measurement Unit Interpretation 13 MONITORING THE AWS EC2 CLOUD of the test Availability Percent Availability failures could be caused by This measurement indicates several factors such as the web server whether the test was able to process es hosting the configured web access the configured URL or page being daw ihe web Ie bring figi misconfigured a network failure etc Temporary unavailability may also occur if the web server is overloaded Availability is determined based on the response code returned by the test A response code between 200 to 300 indicates that the configured web page is available Total response time Secs Response time being high denotes a problem This meas rernent indicates Poor response times may be due to an the time taken by the test to overload If the URL accessed involves the access this URL generation of dynamic content backend problems e g an overload at the application server or a database failure can also result in an increase in response time Tcp connection availability Percent Failure to establish a TCP connection may a imply that either the web server process ee ee Ate hosting the web page is not up or that the esta
80. ot be able to access the server instances launched in that region This may in turn adversely impact the user experience with the cloud To avoid such an unpleasant outcome it is best to periodically monitor the availability of each region so that unavailable regions can be quickly and accurately identified and the reasons for their non availability remedied 15 MONITORING THE AWS EC2 CLOUD This test performs periodic availability checks on each region on the cloud and reports the status of the individual regions In addition the test also indicates the time taken for connecting to a region so that regions with connectivity issues can be isolated Purpose Performs periodic availability checks on each region on the cloud and reports the status of the individual regions In addition the test also indicates the time taken for connecting to a region so that regions with connectivity issues can be isolated Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test 16 MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http
81. particular server choose the On option To disable the capability click on the Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the following conditions are fulfilled e The eG manager license should allow the detailed diagnosis capability e Both the normal and abnormal frequencies configured for the detailed diagnosis measures should not be 0 Outputs ofthe One set of results for each instance launched by the configured AWS user account in the test monitored region Mens uremens Measurement Mossurument Interpretation made by the Unit P test Has the instance Boolean If this measure shows 1 it means that the been rebooted instance was rebooted during the last Indicates whether this instance has been rebooted during the last period or not measurement period By checking the time periods when this metric changes from 0 to 1 an administrator can determine the times when this measurement instance was rebooted 72 MONITORING THE AWS EC2 REGION Uptime of the Secs If the instance has not been rebooted during the instance during the last measurement period and the agent has been last measure running continuously this value will be equal to period the measurement period If the instance was Indicates the time rebooted during the last measurement period this period that the value will be less than the measurement period o
82. performed on this instance Disk write Operations Sec operations Indicates the rate at which disk write operations were performed on this instance These measures are good indicators of the level of disk I O activity on an instance By comparing the values of these measures across instances you can accurately determine which instance is performing I O intensive operations 3 3 3 EC2 Instance Uptime Test In cloud based environments it is essential to monitor the uptime of server instances launched on the cloud By tracking the uptime of each of the instances administrators can determine what percentage of time an instance has been up Comparing this value with service level targets administrators can determine the most trouble prone areas of the infrastructure hosted on the cloud In some environments administrators may schedule periodic reboots of their instances By knowing that a specific instance has been up for an unusually long time an administrator may come to know that the scheduled reboot task is not working on an instance This test monitors the uptime of each instance available to the configured AWS user account Purpose Monitors the uptime of each instance available to the configured AWS user account in the monitored region Target of the Amazon EC2 Region test Agent A remote agent deploying the test 70 MONITORING THE AWS EC2 REGION Configurable parameters
83. port resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to rue On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example j b0c3e 7dbe56d By default this parameter is set to none 33 MONITORING THE AWS EC2 CLOUD 10 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its reg
84. r and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for each type of instance launched by the configured AWS user account test Measurements Measurement Measurement p Interpretation made by the Unit test CPU utilization Percent A high value for this measure indicates that Indicates the one more instances of a type are utilizing CPU percentage of excessively this could be because of one more allocated CPU resource intensive processes executing on the
85. r and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG system embeds an optional detailed diagnostic capability With this capability the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a particular server choose the On option against DETAILED DIAGNOSIS To disable the capability click on the Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the follo
86. r example an AMI might contain all the software to act as a web server e g Linux Apache and your web site or it might contain all the software to act as a Hadoop node e g Linux Hadoop and a custom application After an AMI is launched the resulting running system is called an instance All instances based on the same AMI start out identical and any information on them is lost when the instances are terminated or fail Users with valid AWS user accounts can sign into the EC2 cloud to view and use available instances or purchase and launch new ones With the help of this test you can determine the total number of instances that are currently available for the configured AWS user account the number of instances that were newly purchased terminated and the count of powered off instances Purpose Determine the total number of instances that are currently available for the configured AWS user account the number of instances that were newly purchased terminated and the count of powered off instances Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test 22 MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain th
87. r standard applications whereas others are designed for CPU intensive applications or memory intensive applications etc The different instance types offered by the AWS EC2 cloud are as follows 61 MONITORING THE AWS EC2 REGION Um SPU Mem tois 1 EC2 Compute Unit 1 virtual core with 1 EC2 Compute Unit 4 EC2 Compute Units 2 virtual cores with 2 EC2 Compute Units each 8 EC2 Compute Units 4 virtual cores with 2 EC2 Compute Units each 160 GB instance storage 150 GB plus 10 GB root partition 850 GB instance storage 2 x 420 GB plus 10 GB root partition storage 4 x 420 GB plus 10 GB root partition High CPU Medium High CPU Extra Large High Memory Extra Large High Memory Double Extra Large High Memory Quadruple Extra Large Up to 2 EC2 Compute Units for short periodic bursts 5 EC2 Compute Units 2 virtual cores with 2 5 EC2 Compute Units each 20 EC2 Compute Units 8 virtual coms with 2 5 EC2 Compute Units each 6 5 EC2 Compute Units 2 virtual coms with 3 25 EC2 Compute Units each 13 EC2 Compute Units 4 virtual coms with 3 25 EC2 Compute Units each 26 EC2 Compute Units 8 virtual cores with 3 25 EC2 Compute Units each EBS volumes for storage 350 GB instance storage 340 GB plus 10 GB root partition 1690 GB instance storage 4 x420 GB plus 10 GB root partition 420 GB instance storage 1 x 420 GB 850 GB instance stora
88. rameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the cloud If you want these tests to report resource usa
89. rry 49 3 2 1 EC2 Instance BU sanuME m 50 3 2 2 BG Instance Connectivity Testa oe oen edite He nr ee RHET e EHE DUST Tert E cent decre Desa 53 3 2 3 EC2 Instances est hte tree irre e e be Fe ERES EE REX HE TERRE ESS M LRRERE sansa FEE EGER HERREN ERR ERE RRER 56 3 3 The AWS EC2 Region Instance Details Layer eese eene nennen nennen nennen enne 60 3 3 1 EC Aggregated Resource Usage 6est neret rr re ek ETHER NS RREE SER KEY EU Ye REESE SEU RER SEHE EQ EE X CER ERR 61 3 3 2 EC2 Instance Resources Test entrer eant rb terree cons reine beo e arator reste ri npe reso P reser erae edo 66 3 3 3 EC2 Instance Uptime T68t uenerit ens RE enim ese mue vedete tiber ine ete A ttr cs 70 CONCLUSION Dec 74 Figure 1 1 Figure 2 1 Figure 2 2 Figure 2 3 Figure 2 4 Figure 2 5 Figure 3 1 Figure 3 2 Figure 3 3 Figure 3 4 Figure 3 5 Figure 3 6 Figure 3 7 Figure 3 8 Figure 3 9 Table of Figures How eG monitots the cloud dee ee ae b re pa e o D P pev Re s 2 Layer model of the AWS EC2 Cloud The test associated with the Hardware layer Regions and Availability zones The tests mapped tothe AWS Cloud VM Status layer sess enne nne tns tnnt nne tnt tnter enne e nne nnnnne 19 The tests mapped to the AWS Cloud VM Details layer esses nhennetnnten tnnt nn enne tne nete tenen ene inne tene 25 The layer model of the AWS E
90. s all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default 51 MONITORING THE AWS EC2 REGION 11 PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none Outputs of the One set of results for the AWS EC2 region being monitored test ensure niente Measurement Measurement Interpretation made by the Unit P test Was instance If the instan
91. s the username and password have to be provided as part of the POST information and NOT as part of the CREDENTIALS specification for the this test PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none TIMEOUT Here specify the maximum duration in seconds f
92. s document has clearly explained how eG Enterprise monitors the AWS EC2 cloud and region For more information on eG Enterprise please visit our web site at www eginnovations com or write to us at sales eginnovations com 74
93. s measure validates whether the test was successful in executing the request made to it Percent A value of 10096 indicates that the content returned by the test is valid A value of 0 indicates that the content may not be valid This capability for content validation is especially important for multi tier web applications For example a user may not be able to login to the web site but the server may reply back with a valid HTML page where in the error message say Invalid Login is reported In this case the availability will be 100 since we got a valid HTML response If the test is configured such that the content parameter should exclude the string Invalid Login in the above scenario content validity would have a value 0 2 2 3 AWS EC2 Regions Test Amazon EC2 provides the ability to place instances in multiple locations Amazon EC2 locations are composed of Availability Zones and Regions Regions are dispersed and located in separate geographic areas US EU etc Each Region is completely independent By launching instances in separate Regions you can design your application to be closer to specific customers or to meet legal or other requirements f LR bh UF es w 3 e Amazon EC2 ae vellability zo amp fe ig P B Regions eu wost 1 Avai gt ny Figure 2 3 Regions and Availability zones If a region is unavailable then users to that region will n
94. s model is mapped to tests that reveal the availability of the cloud and whether the regions availability zones instances on the cloud are accessible Using these statistics cloud administrators can find quick and accurate answers for the following critical performance queries a Is web based HTTP HTTPS access to the cloud available b Does it take an unreasonably long time to establish contact with the cloud Cc How many regions does the cloud support What are they d Is any region unavailable e Were any connectivity issues experienced while attempting to connect to a region If so which region is this f How many availability zones exist in each region What are they g Is any availability zone currently unavailable If so which one is it h Is the default region on the cloud accessible If so is it taking too long to connect to the default region i Are all instances on the cloud accessible over the network j Are any instances powered off currently MONITORING THE AWS EC2 CLOUD k Were any instances launched removed recently If so which ones are these What type of instances are resource intensive m Is any particular instance consuming too much CPU n Is the network traffic to zofrom any instance unusually high o Is the disk I O of instances optimal p Was any instance rebooted recently If so which one is it To enable the eG agent to collect these useful metrics the following pre requisite
95. s need to be fulfilled e The eG agent should be deployed on a remote Windows host in the environment e The eGurkhaAgent service of the remote agent should run using domain administrator privileges To know how to set this up refer to the eG User Manual e Each test executed by the remote agent uses the AWS API to collect the required metrics To enable the tests to access the AWS API you need to configure the tests with the access key and password of a user with a valid AWS user account To obtain this access key do the following o o Sign up for a new AWS account from the http aws amazon com ec2 page Provide the details of the user for whom you wish to create the AWS account Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 Once the payment is made the user will be automatically signed in to the AWS account From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key MONITORING THE AWS EC2 CLOUD Note The eG agent reports metrics for only those regions availability zones and instances on the cloud that the configured AWS user account is allowed to access e Some tests require the AWS CloudWatch service to be enabled This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on d
96. s the rate at comparing the values of these measures across which disk read types you can accurately determine the type of operations were instances that is performing I O intensive performed on the disks operations of all instances of this type Disk write Operations Sec operations Indicates the rate at which disk write operations were performed performed on the disks of all instances of this type 2 4 2 Tracking the CPU usage disk and network I O of every instance launched by a configured AWS user account will provide administrators with valuable insights into how well the instances are utilizing the allocated resources The AWS ECS VM Resource Usage test does just that This test auto discovers the instances available for the configured AWS user account and reports the resource usage of each instance so that administrators can quickly compare the usage metrics across instances and pinpoint which instance is resource hungry AWS EC2 Instance Resources Test 31 MONITORING THE AWS EC2 CLOUD Purpose Auto discovers the instances available for the configured AWS user account and reports the resource usage of each instance so that administrators can quickly compare the usage metrics across instances and pinpoint which instance is resource hungry Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test 32 MONITORING THE AWS EC2 CLOUD Conf
97. saved locally and returned with subsequent requests PROXYHOST The host on which a web proxy server is running in case a proxy server is to be used PROXYPORT The port number on which the web proxy server is listening PROXYUSERNAME The user name of the proxy server PROXYPASSWORD The password of the proxy server CONFIRM PASSWORD Confirm the password by retyping it here CONTENT Is a set of instruction value pairs that are used to validate the content being returned by the test If the CONTENT value is none none no validation is performed The number of pairs specified in this text box must be equal to the number of URLs being monitored The instruction should be one of Jnc or Exc Inc tells the test that for the content returned by the test to be valid the content must include the specified value a simple string search is done in this case An instruction of xc instructs the test that the test s output is valid if it does not contain the specified value In both cases the content specification can include wild card patterns For example an Inc instruction can be Inc Home page An Inc and an Exc instruction can be provided in quick succession in the following format Jnc Home Page Exc home CREDENTIALS The HttpTest supports HTTP authentication The CREDENTIALS parameter is to be set if a specific user name password has to be specified to login to a page Against this parameter the URLname of every configured url wil
98. se parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG system embeds an optional detailed diagnostic capability With this capability the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a particular server choose the On option against DETAILED DIAGNOSIS To disable the capability click on the Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the following conditions are fulfilled e The eG manager license should allow the detailed diagnosis capability e Both the bad and normal frequencies configured for the detailed diagnosis measures should not be 0 Outputs of the One set of results for each availability zone in each region of the AWS EC2 Cloud being test monitored Measurements Measurement Measurement Interpretation made by the Unit P MONITOR
99. stens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default these parameters are set to none indicating that the proxy sever does not require authentication by default PROXY DOMAIN and PROXY WORKSTATION If a Windows NTLM proxy is to be configured for use then additionally you will have to configure the Windows domain name and the Windows workstation name required for the same against the PROXY DOMAIN and PROXY WORKSTATION parameters If the environment does not support a Windows NTLM proxy set these parameters to none DETAILED DIAGNOSIS To make diagnosis more efficient and accurate the eG system embeds an optional detailed diagnostic capability With this capability the eG agents can be configured to run detailed more elaborate tests as and when specific problems are detected To enable the detailed diagnosis capability of this test for a particular server choose the On option against DETAILED DIAGNOSIS To disable the capability click on the Off option The option to selectively enable disable the detailed diagnosis capability will be available only if the following conditions are fulfilled e T
100. the Total instances measure 59 MONITORING THE AWS EC2 REGION The detailed diagnosis capability of the 7nstances powered on measure if enabled shows the details of all the powered on instances available for use for the configured AWS account in the monitored region Component myec2 esst region Measured By ec2remote Test EC2 Instances Measurement Instances powered on E Timeline 1hour EJ From B 09 07 11 He i To osor He Details of Instances powered on Instance IP Address Monitoring 09 07 11 02 32 27 zap_mware i bOc3efe2 ami 93ec93ci 122 248 198 156 windovs mi small ap southesst is enabled zap db i bOc3efe3 ami 93ec93c2 122 248 198 164 vindovs mi small ap southeast 1s enabled zap db i bOc3efes ami 93ec93c4 122 248 198 106 windovs mi small ap southeast ia enabled zep_mwere i bOc3efes ami 93ec93c5 122 248 198 225 windows mi small ap southeast 1a enabled Figure 3 6 The detailed diagnosis of the Instances powered on measure The detailed diagnosis capability of the Jnstances powered off measure if enabled shows the details of all the powered off instances available for the configured AWS account Detailed Diagnosis Measure Graph Summary Graph Trend Graph Fix History Fix Feedback Component myec2 east region Measured By ec2remote Test EC2 Instances Measurement nsta Details of Instances powered off Time Name Instance AMIID IP Address Type Monitoring 09 07 11 02 42 08 zap
101. the cloud If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 68 MONITORING THE AWS EC2 REGION 10 PROXYHOST and PROXY PORT In some environments all communic
102. the test reports whether the configured AWS user account is able to access the cloud based infrastructure or not and if so how quickly the connection with the infrastructure was established Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being configured AWS ACCESS KEY To monitor an AWS EC2 the eG agent has to be configured with the access key of a user with a valid AWS account To obtain the access key follow the steps given below e Sign up for a new AWS account from the http aws amazon com ec2 page e Provide the details of the user for whom you wish to create the AWS account e Based on the AWS EC2 Regions you will be requested to choose the pricing for an instance to be deployed in the AWS EC2 e Once the payment is made the user will be automatically signed in to the AWS account e From the newly created AWS account you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding t
103. torage 1 x 420 GB 850 GB instance storage 1 x840 GB plus 10 GB root partition 1690 GB instance 64 bit storage 2 x840 GB plus 10 GB root partition Cluster GPU 33 5 EC2 Compute 23 GB Units 2 x Intal Xeon X5570 quad core Nehalem architecture 33 5 EC2 Compute 22 GB Units 2 x Intal Xeon see note X5570 quad core after this Nehalem architecture plus 2 NVIDIA Tesla M2050 Fermi GPUs 1690 GB instance 64 bit 64 bit storage 2 x 840 GB plus 10 GB root partition 1690 GB instance 64 bit storage 2 x 840 GB plus 10 GB root parlition 27 MONITORING THE AWS EC2 CLOUD By closely monitoring the CPU usage and the network and disk I O of each instance type and comparing these metrics across instance types you can quickly isolate resource intensive types Once again the test will report metrics for only those types of instances that were launched by the AWS user account configured for the test Purpose Closely monitors the CPU usage and the network and disk I O of each instance type and enables usage comparison across instance types so as to quickly isolate resource intensive types Target of the Amazon EC2 Cloud test Agent A remote agent deploying the test 28 MONITORING THE AWS EC2 CLOUD Configurable parameters for the test TEST PERIOD How often should the test be executed HOST The host for which the test is being c
104. u can modify this default setting to configure any URL of your choice eg the URL of the login page to your cloud based infrastructure 3 HOST The host for which the test is to be configured 4 PORT The port to which the specified HOST listens 5 COOKIEFILE Whether any cookies being returned by the web server need to be saved locally and returned with subsequent requests 6 PROXYHOST The host on which a web proxy server is running in case a proxy server is to be used 7 PROXYPORT The port number on which the web proxy server is listening 8 PROXYUSERNAME The user name of the proxy server 9 PROXYPASSWORD The password of the proxy server 10 CONFIRM PASSWORD Confirm the password by retyping it here 11 CONTENT Is a set of instruction value pairs that are used to validate the content being returned by the test If the CONTENT value is 7one none no validation is performed The number of pairs specified in this text box must be equal to the number of URLs being monitored The instruction should be one of Jnc or Exc Inc tells the test that for the content returned by the test to be valid the content must include the specified value a 12 MONITORING THE AWS EC2 CLOUD 12 13 14 15 16 simple string search is done in this case An instruction of Exc instructs the test that the test s output is valid if it does not contain the specified value In both cases the content specificati
105. ue could indicate powered on a problem situation that requires further Indicates the time investigation taken for the configured instance to be powered on Note that these measures will report values only if the corresponding operation succeeds For Time taken for Secs instance the Time taken for instance to be instance to be powered on measure will report valid metrics only powered off if the Was instance powered on successful Indicates the time measure reports the value Yes taken for the configured instance to be powered off 3 2 2 EC2 Instance Connectivity Test Sometimes an instance could be in a powered on state but the failure of the operating system or any fatal error in internal operations of the instance could have rendered the instance inaccessible to users In order to enable you to promptly detect such hidden anomalies this test periodically runs a connectivity check on each instance available for the configured AWS user account in the monitored region and reports whether the instances are accessible over the network or not Purpose Runs a connectivity check on each instance available for the configured AWS user account in the monitored region and reports whether the instances are accessible over the network or not Target of the An AWS EC2 Region test Agent A remote agent deploying the test 53 MONITORING THE AWS EC2 REGION Configurable pa
106. uld result Indicates the average delay from misconfiguration of the router s along between transmission of the path network congestion packets to this instance retransmissions at the network etc and receipt of the response to the packet at the source Min network delay Secs A significant increase in the minimum round The minimum time trip time is often a sure sign of network between transmission of a congestion packet and receipt of the response back Packet loss Percent Packet loss is often caused by network buffer Indicates the percentage of overflows at a network router or by packet packets lost during corruptions over the network The detailed transmils ion from source to diagnosis for this measure provides a listing target and back of routers that are on the path from the external agent to target server and the delays on each hop This information can be used to diagnose the hop s that could be causing excessive packet loss delays Network availability of Percent A value of 100 indicates that the instance is Instance accessible over the network The value 0 Indicates whether the indicates that the instance is inaccessible network connection to this Typically the value 100 corresponds to a instance is available or not Packet loss of 0 21 MONITORING THE AWS EC2 CLOUD 2 3 2 AWS EC2 Instances Test An Amazon Machine Image AMI contains all information necessary to boot instances of your software Fo
107. unt you can request for an access key You will be provided with an access key and a corresponding secret key Provide the access key in the AWS ACCESS KEY text box this will enable the eG agent to communicate with the AWS API and collect the required metrics CONFIRM PASSWORD Confirm the password by retyping it here AWS SECRET KEY Provide the secret key corresponding to the access key that you had obtained through your AWS account CLOUDWATCH ENABLED This flag applies to the EC2 Instance Resources and EC2 Aggregate Resource Usage tests only These tests report critical metrics pertaining to the resource usage of the server instances launched in the monitored region If you want these tests to report resource usage metrics very frequently say once every minute or lesser you will have to configure the tests to use the AWS CloudWatch service This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to rue On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to fa se Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web
108. urces EC2 Instance Uptime and EC2 Instance Deployment tests In the EXCLUDE INSTANCE text box provide a comma separated list of instance names or instance name patterns that you do not wish to monitor For example j b0c3e 7dbe56d By default this parameter is set to none REPORT INSTANCE DATACENTER By default this test reports the availability of only those availability zones that contain one more instances Accordingly this flag is set to true by default If you want the test to report metrics for all availability zones regardless of whether not they host instances set this flag to false MONITORING THE AWS EC2 CLOUD 10 11 12 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy server and collects metrics To enable metrics collection via a proxy specify the IP address of the proxy server and the port at which the server listens against the PROXY HOST and PROXY PORT parameters By default these parameters are set to none indicating that the eG agent is not configured to communicate via a proxy by default PROXY USERNAME and PROXY PASSWORD If the proxy server requires authentication then specify a valid proxy user name and password in the PROXY USER NAME and PROXY PASSWORD parameters respectively By default the
109. urement period is 300 secs and if the instance was rebooted 120 secs back this metric will report a value of 120 seconds The accuracy of this metric is dependent on the measurement period the smaller the measurement period greater the accuracy Total uptime of the instance Indicates the total time that this intance has been up since its last reboot Mins Administrators may wish to be alerted if an instance has been running without a reboot for a very long period Setting a threshold for this metric allows administrators to determine such conditions 38 MONITORING THE AWS EC2 REGION Monitoring the AWS EC2 Region Amazon EC2 provides the ability to place instances in multiple locations Amazon EC2 locations are composed of Availability Zones and Regions Regions are dispersed and located in separate geographic areas US EU etc Availability Zones are distinct locations within a Region that are engineered to be isolated from failures in other Availability Zones and provide inexpensive low latency network connectivity to other Availability Zones in the same Region By launching instances in separate Regions you can design your application to be closer to specific customers or to meet legal or other requirements The AWS EC2 Region model offered by eG Enterprise monitors a specific region on the cloud and reports the availability and responsiveness of that region AWS EC2 Region Instance Deta
110. vice This is a paid web service that enables you to monitor manage and publish various metrics as well as configure alarm actions based on data from metrics To enable the above mentioned tests to use this service set the cloudwatch enabled flag to true On the other hand to report resource usage metrics less frequently say once in 5 minutes or more these tests do not require the AWS CloudWatch service in this case therefore set the cloudwatch enabled flag to false Note that for enabling CloudWatch you will have to pay CloudWatch fees For the fee details refer to the AWS EC2 web site EXCLUDE INSTANCE This parameter applies only to EC2 Instance Connectivity EC2 Instance Resources EC2 Instance Uptime and EC2 Instance Deployment tests Since these tests report metrics for each server instance launched on the cloud you can optionally configure these tests to exclude one more server instances from monitoring For this specify a comma separated list of instance names or instance name patterns that need not be monitored in the EXCLUDE INSTANCE text box For example b0c3e 7dbe56d By default this parameter is set to none 29 MONITORING THE AWS EC2 CLOUD 10 PROXYHOST and PROXY PORT In some environments all communication with the AWS EC2 cloud and its regions could be routed through a proxy server In such environments you should make sure that the eG agent connects to the cloud via the proxy serve
111. wing conditions are fulfilled e The eG manager license should allow the detailed diagnosis capability e Both the bad and normal frequencies configured for the detailed diagnosis measures should not be 0 Outputs of the One set of results for the AWS EC2 Region being monitored test CEST Measurement Measurement Interpretation made by the Unit P test Total instances Number The detailed diagnosis capability of this measure Indicates the total number of instances currently available for the configured AWS user account in the monitored region if enabled shows the details of all the instances available for use for the configured AWS account regardless of the current state of the instances 58 MONITORING THE AWS EC2 REGION Instances powered Number The detailed diagnosis capability of this measure on if enabled shows the details of all the powered on instances available for use for the configured AWS account Indicates the total number of instances that are currently powered on in the monitored region Instances powered Number The detailed diagnosis capability of this measure off if enabled shows the details of all the powered Indicates the total off instances available for the configured AWS number of instances account that are currently powered off in the monitored region Added instances Number The detailed diagnosis capability of this measure Indicates th

Monitoring the AWS EC2 Cloud

Contents

Download Pdf Manuals

Related Search

Related Contents