Test Automation–Success or Failure?
Contents
1. LN If your testing process is disorganized or development Of AUT happens in Chaotic process then any tool is not going to solve the problem for you rather they create more problems Elfriede Dustin Effective Software Testing Zensar TCoE Automation Success or Failure Page 10 ZenSar TECHNOLOGIES L Thought 7 ct How effectively we use a purchased tool depends on how effectively the tool fits our project purpose Cem Kaner Lessons Learned in Software Testing Zensar TCoE Automation Success or Failure Page 11 ZenSar su 9 ui B n your project whatever automation you did till now List down the below Do testers waste time Yes how much No Never correcting script just to approximately make them work for a specific test in a specific environment Did scripts had Hard Yes too much No not at all coding How many times script Every time Some times fail during execution Zensar TCoE Automation Success or Failure Page 12 Why worry About Automation Zensar TCoE Automation Success or Failure Page 13 ZenSar TECHNOLOGIES E Automation is Special because b ms OO Is spent on Tools Training Resources Upfront even before getting automation deliveries Zensar TCoE Automation Success or Failure Page 14 ZenSar TECHNOLOGIES E Automation is Special because 1 PO o It r2. name of automation What did you reply Zensar TCoE Automation Success or Failure Page 19 ZenSar TECHNOLOGIES A Did you reply Yes Tt s a sad part of story I cannot help X was using this tool and he left this company Y has taken over and says lot of project functionality is not captured by tool the tool vendors support is also not responsive we did record AUT but when tried to play we see errors here and there but let me tell you scripts work in SQE environment not in stage Seems lot of hard coding is there That is the reason we are still using manual methods to do testing Zensar TCoE Automation Success or Failure Page 20 Zen Sar TECHNOLOGIES T Difficult times THese are difficult times there is a High pressure on M Producing more with less resources E Optimum utilization of budget E Control on spendisse M Cut down cost E Beat competition IH Ww If your project is one which is Highly Visible Highly Important then management has every reason to Question you If there is no output on automation Zensar TCoE Automation Success or Failure Page 21 ZenSar TECHNOLOGIES 7 Qu IZ nnm az x In last one year or so Did you buy any automation tool In your organization If yes list down how many projects are using this tool Which project Not using this tool and why Did you hear any mandate from management guys We have to use this tool because we h
3. Automation Success or Failure saa because all tools come with capabilities and drawbacks By Govind Kulkarni CSQA PMP Step Auto Forum Test Automation Conference Hotel Pride Executive Pune m d 30 July 2008 ZenSar Your Transformation Partne Thoughts on Automation mWhy worry about Automation Project mHow to Judge Success or Failure HSome Examples and Quiz EWhy Automation Fails A fool with a Zensar TCoE Automation Success or Failure Page 2 ZenSar ZenSar TECHNOLOGIES a Thought 1 x Software development is still labor intensive process even today Although in recent past we had tools available in the market but the fact remains and believe me its still labor intensive Watts S Humphrey Managing Software Process Zensar TCoE Automation Success or Failure Page 4 ZenSar TECHNOLOGIES Thought 2 hen we do not feel like using a tool A Hf the task is one off a nature B No tool needed H If the task is not complex B No tool needed H If the task takes relatively small effort B No tool needed Zensar TCoE Automation Success or Failure Page 5 ZenSar TECHNOLOGIES al Thought 3 When we feel like using a tool HWVhen there is a repetitive work ETo reduce effort ETo increase productivity ETo reduce human strength ETo minimize human mistakes MDo the task better Automation Success or Failure Page 6 Ze
4. NOLOGIES al Some Symptoms LN B Set unrealistic expectations HDemand 100 automation ENo manual Testers needed syndrome BE Wrong tool selection iE Wrong framework selection ENo Training to resources HRely totally on Capture playback Lack of standard Hard to maintain scripts Zensar TCoE Automation Success or Failure Page 32 ZenSar TECHNOLOGIES formation Partne Automation Some Points to Consider Zensar TCoE Automation Success or Failure Page 33 ZenSar TECHNOLOGIES ds While selecting a tool do not limit selection to one single project instead look at organizational requirements Do not demand for 100 Automation Create library of reusable functions Modular and iterative script development Version Control Creating Test Shell or Test Wrapper containing many test scripts is an efficient way to do automation Be knowing less capable light tools Vs Automation goals Zensar TCoE Automation Success or Failure Page 34 ZenSar TECHNOLOGIES formation Partnei ali Thanks Question Answers Contact Details Govind Kulkarni Testing Centre Of Excellence Zensar Technologies Ltd Pune Phone 0091 20 66074626 Cell 9922930376 Email govind kulkarni Zensar com P my aA Zensar TCoE Automation Success or Failure Page 35
5. ave spent money Zensar TCoE Automation Success or Failure Page 22 ZenSar TECHNOLOGIES Your Transformation Partner metum on Investment 3 LL Use ROI as a method to measure success or failure ROI Benefit Vs Cost or Benefit Cost Zensar TCoE Automation Success or Failure Page 23 ZenSar TECHNOLOGIES Return on Investment N E Since the investment is money return also is measured in money ROI How much we get in return for the investment we are making or made measured against Zensar TCoE Automation Success or Failure Page 24 ZenSar TECHNOLOGIES Your Transformation Partner d etum on Investment Deep Good ROI 1 m Show Bad ROI Meter ok Troubl Show Just Ok ROI Meter O Troubl Show ROI Meter Zensar TCoE Automation Success or Failure Page 25 ZenSar TECHNOLOGIES 1000 Test Cases 500 Test Case Automated 4 Test Engineers Total 1500 Test Cases 4 Test Engineers 1 Dedicated and 3 Manual Test Effort 25 days for 4 Test Effort 27 days for people for testing 1000 testing 1500 test cases test cases 1000 manual 500 auto e Say after Six months or so e 27 days as against 37 5 days saving of 10 5days e 10 test cases per day per tester Zensar TCoE Automation Success or Failure Page 26 Zensar TCoE a a pe RESTAR Benefit No of Test Engineers reduced Days saved approx 10 5 by 6 m
6. equires M Application AUT knowledge B Tool and their capability knowledge B Studying and understanding manual test cases B Scripting knowledge E Framework knowledge data keyword hybrid E Testing process knowledge E Knowledge of third party controls e g ActiveX M Knowledge of object orientation B Trouble shooting skills Zensar TCoE Automation Success or Failure Page 15 ZenSar TECHNOLOGIES Your Transformation Partner Zensar TCOE Automation Success or Failure Page 16 ZenSar TECHNOLOGIES Pe for biu guys Do your boss ever call you ask You remember We brought a tool last year spending 10000 Can you tell how we are using it What did you reply Zensar TCOE Automation Success or Failure Page 17 Zen nSar TECHNOLOGIES afl id you reply ex we got the tool but its with IT dept I tried to read user manual but not getting much time I asked him to do a small experimentation but have not heard yet let me check and get back to you I think project X is using this tool but I have no idea how effectively they are using may be we have to set up a meeting to check what they are up to Can we meet next week and discuss this Zensar TCoE Automation Success or Failure Page 18 ZenSar TECHNOLOGIES a a for you guys How often you get into this confronting situation your management says buddy you wasted money in the
7. nSar TECHNOLOGIES gi Thought 4 Automation tools are merely part of the solution H They are not magic answers to testing problems H They cannot replace analytical skills of humans H They are seen as an enhancement to manual testing process H The notion of management that automation tools can do everything in Testing Process is false 4 BWe need multiple tool to support all testing tasks Zensar TCoE Automation Success or Failure Page 7 ZenSar TECHNOLOGIES Your Transformation Partner Different tool Different purpose Code Coverage Analyzers Identify un tested code __ Code Coverage Analyzers 2 Memory Leak Detection Identify memory is managed properly in Application 3 Generate Test Data 9 YA Test Management Tools Manage Test Process Network Testing Tools Test Performance of bandwidth of networks Testing tool Capture and Play back Load Testing Tool Test Load stress of application Metrics Reporting tool Capture measurements and provide metrics B Costly and most frequently used tools Zensar TCoE Automation Success or Failure Page 8 ZenSar TECHNOLOGIES ahs Thought 5 mWhat is the purpose of Test automation E Speed up testing B e g I18N App Compatibility tests Regression B To reduce testing effort E e g 3000 test case 50 testers take 8 days Zensar TCoE Automation Success or Failure Page 9 ZenSar TECHNOLOGIES E Thought 6
8. onths x 136 executing test cases by x3X 10 automation than manual for 4 Test Engineers for 6 months 3000 per tester PM 136 PD PT Automation Success or Failure ZenSar TECHNOLOGIES Data To collect What was strength of Test team six months ago and today Collect data how much time it takes to do testing using manual test case and by using script Rates of resource Page 27 ZenSar TECHNOLOGIES xample Cost ea Sr Description Measure Data To Collect No VR ES 1200 Consider only if paid le PR early licensing fee monitoring defect logging and maintenance of scripts 3 1 Automation Engineers doing 1 x 3000 x 6 The whole cost of automation framework scripting execution months development usually done by dedicated resources Zensar TCOE Automation Success or Failure Page 28 ZenSar TECHNOLOGIES Return on Investment E Make ROI Calculations a simple exercise Too much of math u BBC fx oo or stats means your bosses may not understand what you wanted to say Make ROI easy to calculate and easy to interpret Zensar TCoE Automation Success or Failure Page 29 ZenSar TECHNOLOGIES Sr Qu What was the ROI in our example Zensar TCoE Automation Success or Failure Page 30 ZenSar TECHNOLOGIES Your Transformation Partner Why Automation Fails Zensar TCOE Automation Success or Failure Page 31 ZenSar TECH
Download Pdf Manuals
Related Search