Home

SMOKE TESTING BENCH FOR FREENEST SERVICE

1. Click Link Active Tickets Mine first Page Should Contain List all active tickets by priority Click Link Index by Title Page Should Contain Index by Title Index by Date Click Link WikiStart Page Should Contain Welcome to FreeNEST TRAC Click Link TracGuide Page Should Contain The Trac User and Administration Guide Click Link WikiPageNames Page Should Contain Wiki Page Names Click Link TracPermissions Page Should Contain Trac Permissions Click Link TracWiki Page Should Contain The Trac Wiki Engine Click Element logoutBtn Close All Browsers 61 APPENDIX 5 TESTCASE TO VERIFY THE IRC LOGIN This test case will log user into IRC chat and verify the connection is successful Since the IRC opens up in a pop up window the active window has to be switched twice and a certain frame has to be selected in the IRC window Furthermore during the log in it is recommendable to use the sleep keyword to pause the test to make sure that the user has enough time to connect to the IRC chat room before executing the Current Frame Should Contain keyword x Settings Library SeleniumLibrary xxx Test Cases IrcLogin Open Browser http freenestdemoserver localdomain ff Input Text uname AdminUser Input Password password AdminUser Click Button Login Click Link Communication dont wait Click Link IRC C
2. Selenium Software testing framework used for web applications Cloud computing TestLink Robot Framework HTML Python Jython OpenStack Junkcloud Firebug Git PHP RIDE Cloud computing is a general term used when delivering any locally hosted services over the Internet for external clients Web based test management program A generic test automation framework which development is supported by the Nokia Siemens Network Hypertext Markup Language is a language mainly used to create documents and pages for the World Wide Web Programming language with a design that emphasizes on code readability Implementation of the Python language written in Java which is another programming language Open source cloud computing platform for creating public and private clouds Unofficial name of the OpenStack cloud developed and maintained by the SkyNEST project A plug in for Firefox to get more information about web sites Used as versatile web development tool Open source version control system Programming and scripting language usually used inside HTML pages Tool for creating test cases for Robot Framework Functional testing Non Functional testing Functional tests validate the proper behavior of a certain process in precisely described scenarios Used for estimating the overall performance of a system under testing 1 INTRODUCTION Testing has always been a major
3. Selenium is a set of tools that can be used to automate browsers These tools are primarily intended for testing web applications via automation although their usage is not limited in anyway and can be used in any other tasks as well Selenium can be used individually but it can also be controlled by other testing frameworks It is supported by multiple browser platforms and all its operations are very flexible since many options are given in locating an element in the user interface of the system under test There is a number of different ways to compare test results with the expected results of any driven tests thus accuracy of these tests is not a problem either SeleniumHQ 2012 Selenium RC and WebDriver There are four different software tools in Selenium This suite of tools allows testers to perform automated tests to almost any web application with a vast amount of testing functions Selenium Remote Control Selenium RC was the first main project of Selenium and it is still widely supported however mainly it has been replaced with Selenium WebDriver Selenium 2 Both of these tools are meant for controlling browsers to perform the tests on web applications Even if falling into the shadow of Selenium WebDriver Selenium RC is still being widely used having support for many different languages and internet browsers that the Selenium WebDriver is not compatible with yet Selenium 2012 Selenium IDE Selenium Integrated Devel
4. community cloud public cloud and hybrid cloud NIST 2012 14 Cloud Clients Web browser mobile app thin client terminal emulator SaaS CRM Email virtual desktop communication games PaaS Execution runtime database web server development tools Application Platform laaS Virtual machines servers storage load balancers network Infra structure FIGURE 1 The three cloud service models Wikipedia 2012 There are many benefits for the clients of cloud compute services the most noticeable being the scalability of the service The client organization can add or subtract the amount of resources they need making the service very flexible These services are very easy to implement since the clients do not have to buy their own hardware software or implementation services all is handled by the cloud compute service provider Since all the storaging and server managing is outsourced a company can lessen the burden on its own IT team thus the in house IT department can focus more on the business critical tasks rather than increasing manpower with costly training Lastly the provider s quality of service offers continuous support and in cases of emergency situations an instant response to the fault at question Waxer 2012 Software as a service SaaS Software as a service allows client organizations to access the provider s applications running on a cloud infrastructu
5. Control to execute a certain test but does not bind the Selenium Control and the test to each other which means next time the test is run again it can be done by some other Remote Control selected by the Selenium Hub This way the Hub can divide the tests that are wanted to be run simultaneously equally between the Remote Controls that are available to use and more Remote 26 Controls can be assigned for the Hub whenever needed which makes it largely scalable Selenium Hub can also run several Selenium Controls on one host machine Selenium Grid 2012 Application Specific Selenium Grid Application Agnostic selenese 4 selenese FIGURE 8 Selenium Grid Setup Selenium Grid 2012 4 6 OpenStack OpenStack is an open source cloud computing platform for creating public and private clouds It gives the possibility for any organization to create and offer their own cloud computing services since OpenStack can be implemented on any standard hardware The three major components of the OpenStack project are Compute Object Storage and Imaging Service Compute is the controller used to start up virtual instances and configure the networking Object Storage is used to store objects in a system which is enormously scalable and large by its capacity The Imaging service is a system used to lookup and retrieve the virtual machine images inside the OpenStack cloud OpenStack 2012 There are also two new noti
6. Test Case name Test usage Test type Notes LoginAdminUser Normally logs in the administrator of FreeNEST Correctness testing LoginLowercase Tries to log in the user by using lowercase letters only and verifies that the log in will fail Security testing correctness testing LoginRecentlyLoggedOut UserNoPW Tries to log in recently logged out user without password and verifies that the log in will fail Security testing correctness testing LoginRecentlyLoggedOut UserNoPWSameBrowser Same as above but without restarting the browser Security testing correctness testing LoginWrongPassword Tries to log in user to FreeNEST with incorrect password and verifies that the log in will fail Security testing correctness testing TestLinkNavigation Navigates trough links found in different pages of TestLink Correctness testing TracAdminRights Verifies that only FreeNEST admin has access to administrative tools in Trac Security testing correctness testing TracNavigation Navigates trough links found in different pages of Trac Correctness testing See appendix 4 for more information UserControlAdminRights Verifies that only FreeNEST admin has rights to manage other users of FreeNEST Security testing correctness testing Web2ProjectLogin Log in test for Web2Project inside FreeNEST with correct user credentials Correctness tes
7. Username Password Cut Copy Paste Delete Select All Note The login process Add a Keyword for this Search You may encounter a message certificate Check Spelling This is normal Please accept Inspect Element exceptions according to the instructions given by your browser This is FIGURE 17 FreeNEST login screen By directly right clicking on the textbox wanted to be filled and selecting the Inspect Element option the firebug will automatically show information from the source 39 code related to the textbox as shown in figure 18 A text field for the firebug should be found on the bottom of the internet browser TI lt gt console HTML y CSS Script DOM Net lt gt Edit input lt div lt form login lt div loginbox lt div container lt body lt html fauth php gt lt div gt lt br gt lt label gt Password lt label gt 4 m Done FIGURE 18 Firebug showing source code of a textbox From the source code can be seen that the name of the username textbox is uname Since a password needs to be inserted also the name of that textbox is also required and by inspecting the source code again can be seen that the name of it is password Now that the names of those two textboxes are known the Input Text and Input Password can be instructed to fill them with login credentials Default username and password for the admin in FreeNEST
8. and it was also decided that the tests should perform functional testing on the system and leave the non functional testing outside the assignment The author had previously been part of the development of the FreeNEST project as a tester so he already had some experience about the tests used to verify the functionality of some of its components When deciding which tests are suitable to be run with the current test executioner many aspects had to be considered to see what tests currently fits into the smoke test plan The first thing to consider is to make sure the test is capable of running automatically and getting correct results Creating a test that inspects visual side of the user interface is not possible with the tools used in the test executioner Some tests might have needed deeper knowledge about programming languages that the author did not have thus making them impossible for him to create The length and contents of each test had to be precisely thought out as well Creating too long tests that check the functionality of too many components at one run had to be avoided because the test execution is interrupted immediately when an error has occurred If the test is used to test multiple components and there is error on the first one none of the other components will be tested On the other hand tests should not be too short either For example when testing the user s accessibility into administration tools the test should be ru
9. correctly to run automatically is selected simply a button has to be pressed and the process starts TestLink sends information of the test that has to be run into a PHP script located in the Master Tester This PHP script then sends the information forward into another script that is written in Python The PHP script is working as a translator for TestLink since TestLink doesn t understand Python at all The Python script is the one sending the request for Robot Framework to perform the test With the information the Robot Framework receives it is able to select a right test case file from all the files that are created in advance and stored inside the Master Tester These test files contain all the commands the Robot Framework then sends for the Selenium server that is also running inside the Master Tester Selenium performs the commands it receives from Robot Framework to the target application hosted in the Deep Forest machine After the test has been run or if any step of it has failed Robot Framework creates a log file and sends the result of the test back to the scripts that forwards them to the TestLink The result of the test is then automatically updated to the TestLink database about the test in question The flow of the process is shown in figure 9 however the GIT Version control Repository shown in the figure is not implemented yet 30 Test automation framework in the cloud Team Server FreeNEST 1 3 Automation Bench Ma
10. info they are able to determine if the product really works as intended Smartbear 2012 Some tests might require a tester to repeat same lines of commands hundreds of times Automated tests can be configured to run the same tests many times continuously as many times as needed Automation will also remove the possibility of human errors since automated tests will follow the configured steps precisely every time and will always record detailed results It is also exhausting for the employee in the organization to keep repeating the same tests frequently Smartbear 2012 3 1 2 Restrictions in automation Even though test automation has very noticeable benefits over manual testing it is not always a viable option to use it In certain occasions it might be too hard or even impossible to create automated tests for the product Some parts of the software may change considerably and therefore the automated tests have to be rewritten and if this happens frequently a great deal of time and effort has to be used constantly to keep them updated Creating working test automation will always take some time and if there is not enough of it and the project has a very strict deadline it is better to stick with manual testing Selenium 2012 Some tests might require somewhat different user interaction than a machine can provide and therefore are not possible to be run automatically For example 10 automated tests might not be able to g
11. is AdminUser so they can be added to the next cell from the locator cell In figure 19 the required user information has been added to the grid File Edit Tools Navigate Run Help cobBeEEBAO E WebminLogin Edit Run jo WebminLogin F 7 8128 External Resources WebminLogin Settings gt gt 1 Open Browser http freenestdemoserver localdo ff 2 Input Text uname AdminUser 3 Input Password password AdminUser FIGURE 19 Inserting login credentials for test case 40 In order to get the user logged in login button has to be pressed The Click Button keyword can be used to do it but again the ID of the button is needed Right clicking on that button and selecting the inspect element again will show the part in the source code needed to be inspected Key attributes for buttons are id name and value Since the value is the only one given here see figure 20 it can be used with the Click Button keyword to carry out the login Figure 21 demonstrates the usage of the keyword in this test case A ed we 07 lt gt Edit button lt div lt form login lt div loginbox lt divfcontainer lt body lt html lt label gt Password lt label gt lt input type password name password gt lt br gt 7 console HTMLv CSS Script DOM Net S 2 lt div gt lt form gt N lt m Done FIGURE 20 Source code of a log in button j
12. part of software development and that is a fact that likely will not change in the future either Software testing is process a designed to ensure that a computer code or program does what it is supposed to do and to make sure that the code does not do anything unintended either Testing is carried out to make software reliable and predictable however it can never guarantee the total absence of errors Creating a clear test plan that includes complete and thought out test cases will help in developing software by detecting as high amount of errors as possible during the production Myers 2004 1 2 43 Selecting the amount of testing that should be performed to the software under development is challenging to estimate System testing in particular can be performed until the time or money of the project ends There should always be some acceptance criteria concerning testing and after they are reached it should end the testing cycle Haikala amp M rij rvi 2006 293 2 BASIS OF THE WORK 2 1 Assigner of thesis The thesis was assigned by JAMK University of Applied Sciences more precisely a cloud software project called SkyNEST that is hosted inside the School of Technology at JAMK SkyNEST is a part of TIVIT Cloud Software program which aims to improve the competitiveness of Finnish software development According to the 2009 survey most significant factors of competitiveness are operational efficiency user experience web softwar
13. seconds Page Sh O ld Contain Create New Requirement lick Link New Task age Should Contain Create New Task lick Link New User Story lick Link New Bug age Should Contain Create New Bug C E C Page Should Contain Create New User Story C P C lick Link Product Backlog y w Q u S o d Contain Product Backlog d Contain Business Value Y w Q o u So o age Should Contain This is example of traceable object age Should Contain List all active tickets by priority E P Click Link Active Tickets P Cc lick Link Active Tickets by Version td w Q u PF o E Q Q O ntain grouping results by version age Should Click Link Description Page Should display Click Link Page Should advanced reports Click Link Page Should the automaticall 60 Active Tickets by Milestone Contain grouping results by milestone Accepted Active Tickets by Owner Contain List accepted tickets Accepted Active Tickets by Owner Full Contain demonstrates the use of full row All Tickets By Milestone Including closed Contain complex example to show how to make My Tickets Contain This report demonstrates the use of y set USER dynamic variable
14. two are the ones that actually host the tools used in the automated testing Deep Forest only hosts the target applications of the tests and it is not always the same instance as many Deep Forest instances can be created as needed Team Server is the machine hosting the FreeNEST project platform which in this case is used for test management and execution TestLink is one of the applications installed by default on the FreeNEST and is also the selected test management program used in the FreeNEST project to create test cases and execute them to perform software testing for the development of FreeNEST TestLink has the option to execute tests automatically however it requires an external automation framework to do it 29 Master tester has all the required tools to execute tests requested by the Team Server Robot Framework was selected to function as the automated tester but in order to perform automated tests on web based applications such as FreeNEST it needs the help of Selenium server Robot Framework also needs commands written inside test files that it can understand and these files are also located inside the Master Tester The test cases created in this thesis will be added as part of those files that the Robot Framework will use 5 2 Operating the tester When a test has to be run with this set up the tester will use the TestLink located inside FreeNEST in the Team Server After the desired test that is configured
15. which is used to estimate the overall reliability of the system in present and in the future Developers can then use all the gathered information from the tests results and this estimation model to decide if the software is ready for releasing Stress tests can actually be considered as a variation of reliability tests where system reliability is tested in extreme conditions Khan 2010 3 2 4 Security testing To make a program not accessible for unauthorized users security testing should be performed to the developed system Finding all the major weaknesses of the system helps preventing the access of unauthorized users thus preventing any harm purposely done to the system It is also important to secure the information of authorized users from outsiders Security testing will not only help against attacks but also helps the system to run longer without any major problems Khan 2010 3 3 Cloud Computing Cloud computing is a service that consists of a group of computing resources such as networks servers storages applications and services These resources are then delivered over the web to the customers and typically these services can be divided into three groups which are called software as a service SaaS platform as a Service PaaS and Infrastructure as a Service laaS These three different types of service are illustrated in figure 1 Cloud computing can also be divided by its means of use into four groups private cloud
16. 5 File Edit Tools Navigate Run Help SeoREHE2 S WebminLogin N i A WebminLogin 3 z wae External Resources WebminLogin Settings gt gt Open Browser http freenestdemoserver localdo ff Input Text uname AdminUser Input Password password AdminUser Click Button Login Click Link Administration dont_wait Click Link Nest Server Administration Input Text user adminuser Input Password pass adminuser 9 Click Button Login Page Should Contain Webmin version Saved D Smoketests WebminLogin tt FIGURE 28 Text check in RIDE The current test technically ends in here but few more keywords should be inserted in to make it more complete The first step that should be added is to log out the user from FreeNEST to make sure it does not affect other tests run after this one This can be done by using the log out button located in the upper right corner of FreeNEST Again the firebug can be used to get information about the button a 0 N e lt gt 7 Console HTMLv CSS Script DOM Net lt gt Edit div logoutBtn lt div loggedUser lt div topBar lt body lt html lt frame ntp lt frameset lt html E lt div id topBar gt S E lt div id loggedUser gt lt span gt Gal lt div gt lt div id nestloaa gt lt div gt SSI ra m J Done FIGURE 29 Source code of log out button in FreeNEST As seen in figure 29 the button is not act
17. 6 2 ACASSIGMER OF THESIS iaa SN AAA asa AAA AAA AAA 6 AA sds csccevazsecnsccdescdeveaudenaesdesedvesuedevanedsdedeveaeduoendeadess 7 2 3 Objectives of the study cccccsccecccccsssessssececeeeeeceesesseaeeeceeeesssessaeeeeeeeesseesessnaeeas 7 3 THEORY aa ee 7 3 1 Test AUTO OM ai ed dt dene 7 3 1 1 Benefits of automated testing ccccconocoononnnnoncnnnnnnnnnnoncnnnnnnnnnanonnnnnnnnnnnnanonos 8 3 1 2 Restrictions in automMatiON ccccccccnononacnnnncnnnnnnnnnanonnnnnccnonnnannn nn nnnannnnnanano nes 9 3 2 Testing Techniguss onena t s e teet ttis khvev n eiimaolasala t s elaoiasa va 10 3 21 Correctness MESURA 11 3 2 2 Performance Testing sedeant iaa AA NN Trad eave aiaa 12 3 2 3 Reliability tosting lt suvos ks AA A Ia a TA odiada 13 3 2 4 SECURITY TESTING aasien a ia dde 13 3 3 Cloud COMPUNE cenina aE a R A EAA REA a 13 A TECHNOLOGIES saasta a anti 15 A AsGONEK Al ko TL 15 AD TSU A pics ee cane wae San V ae Knapas da VAR ahaa nsa UA e ye 16 4 3 Robot Era MeWOT teria ainda 19 4A RIDE eae ol tate A II II etten oe 23 SAM entered AM A TNA Enea aa 24 AG OpenStaCK A II 26 A ca bp I E O en nE sun Oa A aa wane TAAS ch n ta A et tania 27 S TEST EXECUTION ia Mamas a tanne kok me pd il 27 BA AUtomated tester AAA 27 5 2 Operating the testouimsisessum sist ia 29 6 CREATION OF THE SMOKE TEST PLAN and ua o pusu au dla kate aN rad r raa RiN sen 31 61 PIaNNING 36 tana Pas ana A aie SSN enna tuddeetelesdae
18. ERIFY THE FUNCTIONALITY OF SEARCH IN FOSWIKI This test case performs a simple search with the search function of FosWiki The search function does not have a button to launch the search therefore Press Key keyword has to be used to launch it 13 is the id cumber for Enter FAK SSCCTINGS AER Library SeleniumLibrary Awe Test Cases rry WikiSearch Open Browser http freenestdemoserver localdomain ff Input Text uname AdminUser Input Password password AdminUser Click Button Login Click Link Knowledge Sharing dont wait Click Link Wiki Input Text quickSearchBox Example Press Key quickSearchBox 13 Sleep 2 Seconds Page Should Contain FaqExample Click A lement logoutBtn Close All Browsers 59 APPENDIX 4 TESTCASE THAT NAVIGATES THROUGH DIFFERENT PARTS OF TRAC This relatively long test case will navigate in Trac project management service and verify that all the links in the main menu are working Settings Library SeleniumLibrary xxx Test Cases TracNavigation Open Browser http freenestdemoserver localdomain ff Input Text uname AdminUser Input Password password AdminUser ick Button Login ick Link Work Collaboration dont wait ick Link Tickets amp Tasking PASO A ick Link New Requirement Set Selenium Speed 0 2
19. Setup Selenium Grid 2012 ooooocccnnoccccncnooocnnccononnnnnonnnnnos 26 FIGURE 9 The test automation framework created inside the Junkcloud 30 FIGURE 10 Creating a new project with RIDE oconnnconoccnnnncnnnononacnnononcnnnononnononnnnnnnn 33 FIGURE 11 Adding a library to a test project cccconocoononnnnoncnonononacnnnnnnnnnnnnnnanonnnnnnnnons 34 FIGURE 12 Creating a test case for a test project in RIDE oonooooccccnccnccononannnnnonnnnnns 34 FIGURE 13 Empty test case in RIDE oooncccnnccncnnnnonconcnnnnnnnnnnnnonccnnnnnonononnnaronnnnnnnononns 35 FIGURE 14 Search Keywords window in RIDE cssccccccccesssssssececeeeesceesessseeeeeeeens 36 FIGURE 15 Missing an argument in keyword ccccconocooonncnnonnnononononnnnnonnnnnnnnan nn nnnonnnnnns 37 FIGURE 16 Filling the required arguments to a keyword cccononocoocncccncnonanoaonnnnnononnnn 37 FIGURE 17 FreeNEST login screen a a a alada 38 FIGURE 18 Firebug showing source code of a textbox ccooocccccononcccnonononcnccanananononnnnos 39 FIGURE 19 Inserting login credentials for test case occcoccconononoonnnnononnnannnananonononnnns 39 FIGURE 20 Source code of a log in button ccccccononooncncnnnnnonnononnnnnnnoncnnnnannnoncnnnnnnnons 40 FIGURE 21 Usage of the keyword Click Button in RIDE oononoooocccncncncnonononanononononns 40 FIGURE 22 The front page of FreeNEST 0oooocccccccnnconononnnnnononnnnnono
20. The same method used before to log into FreeNEST can be used here By inspecting the source code with firebug can be seen that the locator for username field is user and for the password field it is pass The default username and password for Webmin administrator in FreeNEST is admin and like before the login button has to be pressed again The name for it is Login in this case The required keywords and their arguments have been added to the grid in figure 26 43 Login to Webmin You must enter a username and password to login to the Webmin server on freenestdemoserver localdomain Username Password 7 Remember login permanently FIGURE 25 Login screen of Webmin File Edit Tools Navigate Run Help lt o0 amp EHNEHNao WebminLogin ES Edit 1 Run E amp WebminLogin r y n 0 128 External Resources WebminLogin 1 Open Browser http freenestdemoserver localdo ff 2 2 Input Text uname AdminUser 3 Input Password password AdminUser 4 Click Button Login 5 Click Link Administration dont_wait 6 Click Link Nest Server Administration 7 Input Text user adminuser 8 Input Password pass adminuser Login Open file containing tests FIGURE 26 Login information for Webmin in RIDE Next it should be verified that the browser is currently in the front page of Webmin see figure 27 to confirm if the test is successful or if it has fa
21. WWW JAMK FI SMOKE TESTING BENCH FOR FREENEST SERVICE Teemu Ojala Bachelor s Thesis May 2012 Degree Programme in Information Technology Technology ICT Pill alls JYVASKYLAN AMMATTIKORKEAKOULU Es JAMK UNIVERSITY OF APPLIED SCIENCES Pill E N ze N als JYVASKYLAN AMMATTIKORKEAKOULU OPINNAYTETYON Ey JAMK UNIVERSITY OF APPLIED SCIENCES KUVAILULEHTI Tekij t Julkaisun laji P iv m r OJALA Teemu Opinn ytety 02 05 2012 Sivum r Julkaisun kieli 61 Englanti Luottamuksellisuus Verkkojulkaisulupa my nnetty saakka X Ty n nimi SMOKE TESTING BENCH FOR FREENEST SERVICE Koulutusohjelma Tietotekniikka Ty n ohjaaja t LEINO Janne Toimeksiantaja t RINTAMAKI Marko Tiivistelma Opinn ytety n tarkoituksena oli luoda kolmenkymmenen automaattitestin sarja FreeNEST ohjelmiston eri komponenttien testaamiseen FreeNEST on Jyv skyl n ammattikorkeakoulun sis ll toimivan SkyNEST projektin kehitt m selainpohjainen projektit iss k ytett v ohjelmisto joka koostuu useista eri avoimen l hdekoodin ty kaluista Ty ss luotu testisarja pyrkii korvaamaan useasti toistettavien testien manuaalisen suorittamisen automatisoiduilla testeill jotka voidaan ajaa k ytt en Robot Framework nimist testiautomaatioty kalua Testit on tarkoitus ajaa k ytt en hy dyksi SkyNEST projektin omaa OpenStack pohjaista pilvipalvelua jota my s kehitet n t ll hetkell p
22. a pue n File Edit Tools Navigate Run Help GoRERE 2 WebminLogin Edit Run amp WebminLogin El Ey External Resources WebminLogin Settings gt gt 1 Open Browser http freenestdemoserver localdo ff 2 Input Text uname AdminUser 3 Input Password password AdminUser 4 Click Button Login FIGURE 21 Usage of the keyword Click Button in RIDE After the login the FreeNEST will open up on the front page see figure 22 To get into the Webmin login the browser has to be navigated through the top bar located 41 in the upper part of the screen Webmin can be found under the Administration tab and the link to it is called Nest Server Administration n Y Loggedin as Admin User O Dashboard Helpdesk Change password User details Nest Tool Licences Administratior inest portable project platform Welcome Admin User Latest News Welcome to use Portable Project Platform 1 3 16 Nov2011 y j Fm FIGURE 22 The front page of FreeNEST In order to navigate through these two links the keyword Click Link can be used This keyword also reguires a locator which can be found in the source code and is marked with the ID name or href of the link Figure 23 illustrates the required part of the source code In this case the links are identified with href and they are both named as they are seen in the top bar As shown in figure 24 the first click of a lin
23. and keep track of tests that have to be performed on the target software currently being developed Creating a new project is the first step when using TestLink Test Plans are then created along with many singular Test Cases that can later be assigned into these Test Plans The same Test Case can be assigned into any number of Test Plans and one project can consist of many different Test Plans This kind of hierarchical system makes it so much easier for the users to manage different areas of testing in a project Figure 2 displays the front page of TestLink TestLink 2012 17 Say TestLink 1 9 2 Prague Mozilla Firefox e File Edit View History Bookmarks Tools Help n 6 N ss http testlink sourceforge net demo index php TY ce RA testlink Pad A Most Visited Getting Started m Latest Headlines My TestLink 1 9 2 Prague lech Test TestLink 1 9 2 Prague Testlink Administrator admin My Settings Logout Link project Requirement Specification Test Specification Test Execution Test Reports User Management Events 5st JA Test Project SS Tesi Proet T Test Project Current Test Plan New Test Plan Test Project Management Assign User Roles Define Custom Fields Assign Custom Fields Test Plan Keyword Management Test Plan Management Platform Management Builds Releases Assign User Roles User M G Milestone Overview User Management Role Managemen
24. ated tests that will be created in the future by new members in the project 7 2 Future improvements Since the current smoke tests mostly test the correctness and the security areas of the four different possible methods of testing mentioned in chapter 3 2 Testing Techniques including tests that concentrate on testing the performance and reliability of the target software would make the smoke test plan more complete When the OpenStack cloud the project uses gets improved both technically and materially the limits of the test performance can be minimized Getting new hardware on the OpenStack cloud and optimizing it to its fullest will increase the overall performance of the cloud giving the option to replace the target machine as quickly and as often as needed This will allow the adding of tests that make permanent changes on the target machine such as create thousands of users to the database to see the limits of the system Additionally implementing the Selenium Grid see chapter 4 5 makes it possible to uphold multiple test executors at once Each one of them can perform tests on different target machines at the same time adding some flexibility and efficiency to the automated testing however this too requires more performance from the cloud it is executed on On top of these adding automated and more precise test log reporting back to the test management machine and implementing a way to handle 53 test case updates more ef
25. ational Institute of Standards and Technology Referred on April 25 2012 http nist gov itl cloud OneStopTesting n d Black Box Testing Referred on April 26 2012 http www onestoptesting com blackbox testin OpenStack n d OpenStack The Open Source Cloud Operating System Referred on April 29 2012 http openstack org projects Outsourcebazaar 2006 Automated vs Manual testing Referred on April 17 2012 http www outsourcebazaar com index Article AutomatedVsManualTesing html RIDE n d How To guide for RIDE Referred on May 04 2012 https github com robotframework RIDE wiki How To Robot Framework 2012 Robot Framework User Guide Referred on April 11 2012 http robotframework googlecode com hg doc userguide RobotFrameworkUserGu ide html r 2 7 1 55 Selenium 2012 Introduction to Selenium Referred on April 24 2012 http seleniumhg org docs 01 introducing selenium html Selenium Grid n d How it Works Referred on April 30 2012 http selenium rid seleniumhg org how it works html SeleniumHO n d What is Selenium Referred on April 24 2012 http seleniumhg or Smartbear n d Why Automated Testing Referred on April 25 2012 http support smartbear com articles testcomplete manager overview TestLink 2012 TestLink User Manual Referred on April 5 2012 http www teamst org tldoc 1 9 testlink user manual pdf Waxer C n d The Benefits of Cloud Computing Referred on April 25 2012 http
26. ceable components and the first one is called Identity that provides coherent authentication through all OpenStack projects and is fully 27 capable to be integrated with all existing authentication systems The second new component is Dashboard which allows the users of OpenStack to access and provision resources in the cloud through a self service portal OpenStack 2012 4 7 Firebug Firebug is a plug in application for the Mozilla Firefox web browser used as a web development tool It can also be used on different web browsers however it will have limited functionality on any other browser than Firefox It has a wide range of different features that will benefit any web designer or developer and it is designed to reduce the amount of guesswork in web page debugging and can also be used to inspect accurate information about the web page such as page layout and source code McBlain 2011 5 TEST EXECUTION 5 1 Automated tester The SkyNEST project was in a need of test automation framework to increase the testing efficiency done to the FreeNEST project platform Previously all the testing done after each released build of FreeNEST in development had to be tested manually because the project did not yet have enough time or vacant resources to be used on creating an automated test bench that could be used in the software testing The project was moving on fast and the product was tested manually constantly After many member
27. ch as when using keywords that add certain conditions to another keyword For example a keyword Run Keyword and Expect Error can be inserted in front of a normal keyword such as Click Link The list of all the keywords and their explanation can be found in the internet sites of the Libraries or by opening the Search Keywords window visualized in figure 14 from the dropdown lists located on the top of RIDE COS lt all keywords gt Name Source Choose Cancel On Next Confirmation Choose File Click Button Click Element Click Flex Element Click Image Click Link Close All Browsers Close Browser Close Window Name Source Click Button SeleniumLibrary lt test library gt Arguments locator dont_wait Clicks a button identified by locator SeleniumLi SeleniumLi SeleniumLi SeleniumLi SeleniumLi SeleniumLi SeleniumLi SeleniumLi SeleniumLi SeleniumLi 36 Description Cancel will be selected the next time Confirm Ac Inputs the file path into file input field found by Clicks a button identified by locator Click element identified by locator Clicks an element found by locator Clicks an image found by locator Clicks a link identified by locator Closes all open browsers and empties the conne Closes the current browser Closes currently opened pop up window Key attributes for buttons are id name an
28. d value See introduction for details about locating elements and about meaning of dont wait argument FIGURE 14 Search Keywords window in RIDE The first command to perform is to open up an internet browser This is done by using the keyword Open Browser so it has to be inserted into the first row of the grid After the keyword is written the cell next to it will turn in color red see FIGURE 15 This means that the cell reguires some information inside it to make the keyword work By moving the mouse cursor over the red cell it can be seen that the cell is missing an argument In this case URL Uniform Recource Locator has to be given that is wanted the browser to open into 37 RIDE Wehmmi an File Edit Tools Navigate Run Help coPrEABA O AA WebminLogin Edit Run fk WebminLogin sn 81 25 External Resources WebminLogin Settings gt gt Open Browser Missing argument url FIGURE 15 Missing an argument in keyword This test was meant to verify the login in Webmin but first the browser has to be navigated there through FreeNEST FreeNEST is installed with default settings so its address will be the URL used in this cell It can also be specified in the cell next to the red cell which internet browser we want the test to be run in By writing ff in it Robot Framework will understand to execute the test in Firefox browser The missing argum
29. ds ed sooangeteetsasdediucte aa Tae A 31 6 2 Creation of a Robot Framework Test CaS8 oooocconcccnonccononnconanncnnonncnnanannnoncnnnn ccoo 32 A A ea hance ones 49 JA Discussion ON resultar saa asa an saas dna D makua a KAKUN nA 49 7 2 F ture improvements soaren vain in L TIES STA Uiscidecwedesessdiseddsnsevecseas 52 SOURCES titi 54 APPENDIX 1 TESTCASE THAT VERIFIES ADMIN RIGHTS ON HELPDESK 0008 56 APPENDIX 2 TESTCASE FOR TESTING PASSWORD CHANGE coccooccccncccconcconcconcninncccnnnnns 57 APPENDIX 3 TESTCASE TO VERIFY THE FUNCTIONALITY OF SEARCH IN FOSWIKI 58 APPENDIX 4 TESTCASE THAT NAVIGATES THROUGH DIFFERENT PARTS OF TRAC 59 APPENDIX 5 TESTCASE TO VERIFY THE IRC LOGIN eeseeeeeeeeeeeseeceeeeeeeeeeeeeeaeeeaeens 61 FIGURES FIGURE 1 The three cloud service models Wikipedia 2012 cccoocccccconoccccnnnonano 14 FIGURE 2 A Screenshot of TestLink demo uuuusum asemat a a n aa naan n aan naan anaaent 17 FIGURE 3 Test case in HTML format Robot Framework 2012 cooooncccnnoccccncncnncs 20 FIGURE 4 Test case in TSV format Robot Framework 2012 uuusossstn nnen 21 FIGURE 5 Test case in plain text space and pipe formats Robot Framework 2012 22 FIGURE 6 Test case in reST format Robot Framework 2012 oooonnnnnnnonnccnncconnnn 23 FIGURE 7 Normal Selenium Setup Selenium Grid 2012 ccnoococcccncccccccnnooncnnnnnnnnos 25 FIGURE 8 Selenium Grid
30. e open systems security engineering and sustainable development Cloud software ties these factors together as software increasingly moves to the web Cloud Software program especially aims to pioneer in building new cloud business models lean software enterprise model and open cloud software infrastructure J rvinen 2012 2 2 Background for thesis FreeNEST is developed by using agile software development method and therefore many builds are made frequently FreeNEST is a project platform containing many different open source applications and they all have to be tested as well individually as also regarding their performance between each others Doing hundreds of tests manually after each build would be time consuming and a waste of personnel resources Giving the option for testers to run series of tests automatically and many times faster than by manual execution enables the project team to use these previously wasted personnel resources for developing testing methods even further 2 3 Objectives of the study The main objective the thesis is to create an automatic testing environment for the FreeNEST web service instance and a compact testing plan to execute smoke testing for its future releases The smoke test contains thirty relatively quick tests that confirm the functionality of some of the critical functions of FreeNEST These tests are meant to be run as easily and simply as possible by any tester and all the results from the t
31. e library multiple times with different names Cancel 0 OK FIGURE 11 Adding a library to a test project To add a test case for the project simply right clicking the WebminLogin project found at the left side of the user interface where all the components of the project are listed opens up a list with an option to create one or by using key combination ctrl shift N The program will ask a name for the new test case illustrated in figure 12 and again it will be WebminLogin New Test Case Name WebminLogin Give a name for the new test case FIGURE 12 Creating a test case for a test project in RIDE Keywords to the grid found inside the WebminLogin test case can now be added There are also some extra settings on top of the grid as shown in figure 13 but filling them is not required so they are left empty for now By clicking the settings button they can be minimized 35 A RIDE WebminLogin File Edit Tools Navigate Run Help PoREHAE 2 B WebminLogin Edit Run 8 WebminLogin E ei External Resources WebminLogin Documentation Setup Teardown Timeout Template Tags Edit Clear Edit Edit Edit Edit FIGURE 13 Empty test case in RIDE Keywords are added to the leftmost cells of the rows Usually only one keyword is in one row but there are some exceptions su
32. e project Test Cases are put inside Test Suites much like files are put inside folders in a computer therefore it is recommended to create a number of Test Suites to help keeping the possibly hundreds of Test Cases inside the project clearly sorted One good example is to sort Test Cases by their method of execution either manual or automated TestLink 2012 The first thing to do when creating a Test Case is to give it a title which can for example be a short description of the case There is also a description box to write more detailed information about the test All the steps can be written on a textbox reserved for them and there is also space to write down expected results of the test If the settings of the project allow it test importance and the execution type can also be selected Execution type can be either manual or automated Keywords can be assigned to the Test Case to help categorizing or filtering different Test Cases in some other areas of TestLink TestLink 2012 Test Plans Before any tests can be executed at least one Test Plan has to be created for the project Additionally the Test Plan itself requires at least one build A build is a specific release or a version of software Each project is usually made up of many different builds and in TestLink test execution is based on Builds and Test Cases If the project has no builds created for it test execution is disabled Additionally test Cases have to be assigned int
33. eeNEST FreeNEST is a web based project management tool developed in the SkyNEST project which is hosted inside the JAMK University of Applied Sciences FreeNEST consist of several different open source based software tools The set of tests created in this study aims at replacing manual tests that have to be performed repeatedly with automatic tests executable by using a test automation framework called Robot Framework The tests are meant to be executed using the OpenStack cloud developed and maintained by SkyNEST project In the theory part of this study the benefits and restrictions of automated and manual testing are compared and different methods used in testing are presented Additionally the tools and software used in the creation of the tests are introduced in the text and the creation of a single test is explained step by step using the RIDE editor which is the test case editor of Robot Framework The test cases were finished on time and executing them using the virtual machines created in the OpenStack cloud was successful The total coverage of the created tests however remained slightly incomplete due to certain restrictions in the current test execution system Keywords test automation Robot Framework RIDE FreeNEST SkyNEST OpenStack Miscellaneous CONTENTS EG RES O A 2 TABLES a 3 TERMINOLOGY a eee aac bese eip e EE Ea AAEE SAT ARAE AA 3 TINTRODUCTION cda taa 6 D BASIS OP THEWORK a mata Em ttimen dt ms
34. ents have been inserted for the test case in figure 16 f RIDE WebminLogin File Edit Tools Navigate Run Help coPBEABAO WebminLogin Edit Run ik WebminLogin n E ei External Resources WebminLogin Settings gt gt http freenestdemoserver localdo ff FIGURE 16 Filling the reguired arguments to a keyword 38 If run the test were now run a browser would open into the FreeNEST login screen Next the test should insert the login credentials of the FreeNEST admin into this login screen For that the keywords that are called Input Text and Input Password are used here These keywords require an argument called locator inserted to the cell to the right of them First however the IDs of the text boxes in the login screen have to be known and this can be done by inspecting the source code of the page One way is to just right click anywhere on the page and from the list that opens select the View Page Source Using this method is not recommended since the source codes of some sites are very complicated and finding certain information from them can be difficult this way By installing a plug in for the Firefox called firebug and activating it another option appears in the list when right clicking the page This option is Inspect Element and should be found at the bottom of the list which is shown in figure 17 inest portable project platform Please log in
35. er the system gives the required output the tester has all the information about the steps leading from the input to the output White box testing is used in all types of testing integration unit and system testing White box testing is the most reliable method to make sure that all the steps in the test are properly executed Khan 2010 Black box testing Black box testing is another type of correctness testing however it is not limited to only be used in it In Black box testing the tester has no knowledge of the test product s internal structure The test designer simply selects valid and invalid inputs and determines what should be the expected output when the input is given This method of testing is also usable on all types of testing and it is much simpler than white box testing but it does not give as comprehensive results as white box OneStopTesting 2012 12 Gray box testing The combination of white box testing and black box testing is called gray box testing In this method the tester has some knowledge of the internal structure of the product but the testing itself is still done at black box level To create test cases for gray box testing the test engineer has access to the internal data structures and algorithms however the resulted tests are still not as accurate as in white box testing Khan 2010 3 2 2 Performance Testing Performance tests are executed to determine the speed and effectiveness of a
36. ese tables are still edited in a sententious text format and can be edited using any text editor The test data tables are identified based on the text in the first cell at the same time ignoring all the data that is outside these recognized tables An example of a test case in reST format can be seen in figure 6 Robot Framework 2012 23 Setting Value Value Value Library OperatingSystem Variabie Value Value Value S MESSAGE Helio world Test Case Action Argument Argument My Test Documentation Example test Log MESSAGE My Keyword tmp Another Test Should Be Equal MESSAGE Helio world Keyword Action Argument Argument My Keyword Arguments Sipath Sipath N Directory Should Exist FIGURE 6 Test case in reST format Robot Framework 2012 4 4 RIDE Robot Framework IDE RIDE can be used to create and alternate automated tests in a more graphical and distinct environment It uses files that are in HTML TSV or Plain Text format In RIDE keywords can be given to the test case inside a grid regardless of the file type used All the usable keywords from the available libraries are listed inside the search function of RIDE If more libraries are added later all the keywords from those newly added libraries are also found within this search function RIDE can also be used to execute the created tests and overall makes easier to manage test cases for Robot Framework RIDE 2012 24 4 5 Selenium
37. ests will be reported automatically into a web based test management program 3 THEORY 3 1 Test Automation The reason for running tests on any software in development is to locate and eliminate possible errors so they would not appear in the final product Even if software development groups are trying their best to test their products there will always be defects in the delivered software Testing is performed either manually or itis automated by using automated software testing tools By using this automated method effectiveness efficiency and coverage of the testing can be improved and therefore the number of defects that end up in the final product is minimized Smartbear 2012 When the testing is performed manually a tester simply uses the program normally with a computer It includes going through different screens of the application trying out versatile sets of usage and input combinations and then comparing all the received results to the expected behavior of the software In addition the testers have to record their observations based on the output they get from the program in different situations Smartbear 2012 Automated testing tools can perform tests that have been created in advance They are able to perform specified actions and compare results of these actions to the expected behavior of the software After the tests have been run the automated testing tool can report the success of the test or possible err
38. etely untouched that should be used in smoke testing every new version of the FreeNEST Most of the tests that were included in the smoke test plan are either navigational tests that verify that it is possible to access all the different applications and services provided by FreeNEST or they are tests to perform authorization tests in the forms of login tests or verifying that only authorized users have access into administrative tools found in several places inside FreeNEST project platform The minor part of the tests consists of search tests simple integration tests and tests that edit user information Full list of all the created test cases can be seen in tables 1 and 2 TABLE 1 List of created test cases part 1 2 50 Test Case name Test usage Test type Notes BugzillaAdminRights Verifies that only FreeNEST admin has access to administrative tools in Bugzilla Security testing correctness testing BugzillaLogin Verifies that users logged in FreeNEST are also logged into Bugzilla Correctness testing BugzillaSearch Performs a search action in Bugzilla and verifies that results are found Correctness testing CactiLogin Log in test for Cacti tool inside Correctness testing FreeNEST with correct user credentials ChangePassword Changes the password of a user Correctness testing confirms the change and then changes it back to the default See appendix 2 for more info
39. ficiently when needed with the help of the Git version control system the test automation engine will be in near perfect state Some of these improvements are already being researched by other thesis workers and members of the project and will be implemented gradually in the future into the test execution machines 54 SOURCES Caruso J 2011 laaS vs PaaS vs SaaS Article about cloud computing in Network World Referred on April 29 2012 http www networkworld com news 2011 102511 tech argument iaas paas saas 252357 html Haikala amp M rij rvi J 2006 Ohjelmistotuotanto Helsinki Talentum IBM n d Platform as a service PaaS Referred on April 28 2012 http www ibm com cloud computing us en paas html J rvinen J n d Cloud Software Program Referred on May 4 2012 http www cloudsoftwareprogram org cloud software program Khan E 2010 Different Forms of Software Testing Techniques for Finding Errors Referred on April 26 2012 http www ijcsi org papers 7 3 1 11 16 pdf Krishna Training 2011 Unit Integration and System testing Referred on April 26 2012 http www krishnatraining com unit integration and system testin McBlain A 2011 Firebug Guide for Web Designers Referred on May 04 2012 http sixrevisions com tools firebug guide web designers Myers G 2004 The Art of Software Testing Second Edition Hoboken N J John Wiley Sons NIST n d Cloud Computing Program Article by N
40. ge Should Contain Webmin version E KEYWORD SeleniumLibrary Click Element logoutBtn FIGURE 32 Log file of a passed test run by Robot Framework Normally when one step in a test fails the steps after that are not executed This will cause the Close Browser keyword to be ignored too thus adding this keyword to the teardown arguments in the test case settings mentioned before will cause the browser to shut down even after the test fails Another issue to note is that this test was designed to run on a test machine that has Selenium server running all the time To make sure it runs in other environments too adding the keyword Start Selenium Server to the first row of the test grid will launch the server every time the test is run Additionally inserting the keyword Stop Selenium Server at the very last row will ensure that the server is not left running when it is not needed 49 7 RESULTS 7 1 Discussion on results The creation of the whole set of thirty smoke tests was finished in time for the completion of this thesis and for the most part it fulfills the expectations given at the beginning of this project and all the tests are done as variably as possible for the current test execution engine There were still some factors that dropped the possible amount of different types of tests and therefore the overall coverage of the written smoke tests and leave some areas of the different testing techniques almost compl
41. hat dont wait Get Window Names Select Window popUpProjectIRC Click Button Login dont wait Wait Until Page Contains Element fmain timeout 20 FE Select Frame fmain Sleep 10 seconds current frame should contain has joined Close Window Select Window main Click Element logoutBtn Close All Browsers
42. iled There are several methods of doing this and one of them is to make the test check if certain text is found on the page Other ways are such as checking the source code for certain elements or verifying that the current address of the internet browser is in the right place These all can be used as subsequent set of keywords however for this case verifying some text from webmin should be sufficient 44 Administration Work Collaboration Nest Server Administration MistimteImMEskks1s JeIet a a o AET a PIET e ETET A webmin System hostname freenestdemoserver Operating system Ubuntu Linux 10 04 1 Webmin version 1 570 Time on system Sat Apr 14 12 09 19 2012 FIGURE 27 Webmin main menu inside FreeNEST The keyword to check if certain text is found is called Page Should Contain In the cell next to this keyword a string of text has to be inserted and this is the exact text the test will try to find on the current page The text selected should be something that is not too common and should be found only on that certain page or the text might give a false pass status if it finds the text even if it is in a completely different site Of course this is a good example why multiple checks should be performed when not being sure if one is sufficient For this text check the author has selected the Webmin version text string found in the main page of Webmin and it has been added to the test grid as seen in figure 28 4
43. ing few keywords while still designing it The test can be executed by selecting on the run tab above the test grid and clicking on the start button which is shown in figure 31 Before running the test it needs to be made sure the execution profile is correct depending on what programming language is used In this case it is Python so pybot is selected from the dropdown list 47 File Edit Tools Navigate Run Help eoREHE a E WebminLogin Edit E a if WebminLogin e wa External Resources Denion iane Sta 2 Stop Arguments C Only run tests with these tags Skip tests with these tags T a WebminLogin elap Eo Ume A pass 1 fail 0 working directory C Users Teemu command pybot bat argumentfile c WebminLogin PASS 1 critical test 1 passed 6 failed 1 test total 1 passed 8 failed Output c users teemu appdata loca Log c users teemu appdata loca Report c users teemu appdata loca test finished 64 27 12 17 14 36 Cal FIGURE 31 Executing a test in RIDE After the test has been executed the result is immediately shown in the run screen If everything is done right while creating the test and the target application is not defective either the result should be pass More information about the test run can be found by clicking on the Report and log buttons found next to the Start button These logs can give mo
44. ive feedback on the visual errors in the program If the tests do not need to be run very often it is also better to use manual testing Manual testing allows the tester to perform more random testing ad hoc on the program which can help discovering errors that have never been considered even to exist For the employees in the organization to use these automated testing tools it will always require some knowledge of them If there is not enough experience currently available some time has to be invested in researching these tools The last thing to consider is to calculate costs and compare them to possible benefits of the automation Setting up an automated tester is always more expensive than using manual methods in testing however in the long term using the automated tester will be more cost efficient than doing the tests manually over and over again Outsourcebazaar 2006 3 2 Testing Techniques Finding bugs is not the only reason for software testing since the system in whole also has to work as intended Overall quality and the reliability of the developed software has to be measured and this is achieved by using a few different types of testing techniques and they all have their own purpose in the testing process These four types of techniques are called Correctness Testing Performance Testing Reliability Testing and Security Testing Khan 2010 Testing is also divided into three different types of testing depending on
45. k requires an extra argument called dont wait after the locator Inserting arguments will add additional rules to the keywords that are driven Whenever a click link keyword is run the test will wait until current page has loaded in the browser Since clicking the upper tabs of this top bar in FreeNEST does not result in a page load this dont wait argument has to be inserted or the test will wait for the page load forever and will fail in a timeout 42 s y 7 console HTML y CSS Script DOM Net x gt Edit a tabButtonLink lt li Admin abButton lt ul topbarTabs lt div topb pbarMenu lt div topBar lt body lt html S default ui corner top gt lt li id Administration tabBtn class tabButton Administration tabBtn ui state default ui corner top gt ua lt m m Done FIGURE 23 Identifying the link locator Br i File Edit Tools Navigate Run Help coBEABRA O E WebminLogin Edit Run fk WebminLogin E z 8 2 External Resources WebminLogin Open Browser http freenestdemoserver localdo ff 1 2 Input Text uname AdminUser 3 Input Password password AdminUser 4 Click Button Login 5 Click Link Administration dont_wait 6 Click Link Nest Server Administration FIGURE 24 Usage of keyword Click Link in RIDE After these navigation steps the test should now end in the login screen of Webmin which presented in figure 25
46. n with both admin user account and normal user account since one of those two is allowed to access those tools and the other one should not be Of course this test could be 32 divided into two separate tests which however would lower the efficiency of the driven test to half One last thing had to be considered while making these tests was the restrictions of the current test automation framework None of the tests are allowed to make permanent changes to the system under test and should return the target application to the original state that it was in before running the test if possible The reason for this is that the current set up is not yet efficient enough to replace the target instance with a fresh one instantly whenever needed 6 2 Creation of a Robot Framework Test Case This chapter describes the creation of one of the thirty test cases that were added to the smoke test plan for FreeNEST This example only describes a small fraction of all the possible keywords that can be used with Robot Framework More example test cases can be seen in the appendices of this thesis Libraries that the author had in use were the Robot Frameworks own library called Builtln which has basic commands that can be used in any tests and SeleniumLibrary that enables keywords which can be used by the test to give commands for web browsers to make certain actions The minimum requirement for creating test cases for Robot Framework is any kind of te
47. nonnnnnnnnnnnnnnnnnonnnnnnnnnns 41 FIGURE 23 Identifying the link locator oooooccnnooccccnononcnonononanononanannnonanonccononnnnos 42 FIGURE 24 Usage of keyword Click Link in RIDE occccccconononocnnononcnonanonoranononononns 42 FIGURE 25 Login screen of WebMiN cconconoccocnnccncnonononnnnnnnnnnnnnnnnnononnnnnonnnnnnnnnononnnnnnnnnns 43 FIGURE 26 Login information for Webmin in RIDE cccccnnnononocnncnoncnnnononanannnonnnnnns 43 FIGURE 27 Webmin main menu inside FreeNEST cccsccccecssstececseseeeecesseeeeeeeeeaaes 44 FIGURE 28 Text check in RIDE nda 45 FIGURE 29 Source code of log out button in FreeNEST occcconononoccncnoncnnnonnnannnnnononnnns 45 FIGURE 30 Completed test case in RIDE cccccnonononocnnononnnnnononannncnoncnnnnnnnononnnnnnnnnns 46 FIGURE 31 Executing a test in RIDE eiin aa anaiita i iiaia at 47 FIGURE 32 Log file of a passed test run by Robot Framework cccccconocooncnncnnnnnn 48 TABLES TABLE 1 List of created test cases part 1 2 oooocnnococioocncconanononnnononnnonnnanononanononanos 50 TABLE 2 List of created test cases part 2 2 cooooccnocccooocncconanononanononnnonnnanononanononanos 51 TERMINOLOGY FreeNEST A project management tool developed in the SkyNEST project SkyNEST Name of the project hosted inside JAMK University of Applied Sciences Ad hoc Software testing that is performed at random without any planning or documentation
48. o Test Plans to enable executing the tests TestLink 2012 Test Execution As mentioned above two methods of execution are available for the user which are manual and automated When executing the tests manually the user will simply read the description of the written test and follow the steps as accurately as possible 19 After all the steps have been performed the status of the Test Case can be changed by comparing the perceived results by the tester to the expected results written on the Test Case itself The new given status can be failed passed or blocked When TestLink is configured correctly to allow automated test execution a button that launches the test run can be found in the Test Case TestLink itself cannot perform any automated testing however it can send a request into an external testing framework to perform the test and then report the results back to TestLink TestLink 2012 4 3 Robot Framework Robot Framework is described as a generic test automation framework It is used for acceptance testing and acceptance test driven development ATDD It uses simple keywords that act like commands By implementing new test libraries many more keywords become available to use which allows more complex tests and makes it possible to run tests on many different kinds of target software Robot Framework 2012 Tests for the Robot Framework can be made in four different file formats They can be written using the h
49. oUser Admin Click Element logout Close All Browsers Btn 57 APPENDIX 2 TESTCASE FOR TESTING PASSWORD CHANGE This test case will change the password of DemoUser one of the default users in FreeNEST It will then verify the change by logging in again with the new password and in the end will change the password back to the original thus making the test executable multiple times AXE Settings Library SeleniumLibrary Test Cases ChangePassword Open Browser http freenestdemoserver localdomain ff Input Text uname DemoUser Input Password password DemoUser Click Button Login Click Link Change password Input Text current passwd DemoUser Input Password new passwd demo Input Password verify new passwd demo Click Button btnChange dont wait Run Keyword And Ignore Error Confirm Action Run Keyword And Ignore Error Click Element logoutBtn Close Browser Open Browser http freenestdemoserver localdomain ff Input Text uname DemoUser Input Password password demo Click Button Login Click Link Change password Input Text current passwd demo Input Password new passwd DemoUser Input Password verify new passwd DemoUser Click Button btnChange dont wait Click Element logoutBtn Close Browser 58 APPENDIX 3 TESTCASE TO V
50. opment Environment Selenium IDE is a Firefox plugin which can be used to record user actions on a browser All the actions are automatically converted into test script and can and should be manually altered by the user since the recorder is not capable of adding any conditional statements on the test script Selenium IDE is mainly intended just to create a prototype of a test 25 case and it is recommended to use it with either Selenium RC or Selenium WebDriver Selenium 2012 Selenium Grid The last tool in the set is called Selenium Grid When tests are normally executed with the standard Selenium tools controlling more than six simultaneous browsers with the Selenium is not recommended as the Remote Control becomes too unstable It is possible to target the tests to run on different Remote Controls at the same time this however makes it hard to run specific tests in parallelized fashion A set of Remote Controls created in this way is also very hard to manage and alter later Figure 7 below illustrates the operation of a single Selenium Remote Control Selenium Grid 2012 Application Specific i Application Agnostic o E Slow Usually the bottleneci of your Selenium Test Suite FIGURE 7 Normal Selenium Setup Selenium Grid 2012 Selenium Grid uses a new component that will help to manage a number Remote Controls at once The component is called Selenium Hub illustrated in figure 8 It can allocate one Selenium
51. ore empty spaces or a pipe character that is surrounded with spaces Both of these different ways of cell separation are displayed in figure 5 Just like in TSV format asterisks are used to recognize the test data tables in the text file and all text that is written before the first asterisks will be ignored Robot Framework 2012 Settings Library OperatingSystem Variables S MESSAGE Hello world KKK Test Cases My Test Documentation Example test Log MESSAGE My Keyword tmp Another Test Should Be Equal S MESSAGE Hello world HKH Keywords ee My Keyword Arguments S ipath Directory Should Exist Sipath Setting Value Library OperatingSystem Variable Value S MESSAGE Hello world Test Case Action Argument My Test Documentation Example test Log MESSAGE My Keyword tmp Another Test Should Be Equal MESSAGE Keyword My Keyword Arguments ipath Directory Should Exist ipath 22 Hello worid FIGURE 5 Test case in plain text space and pipe formats Robot Framework 2012 ReStructuredText Format ReStructuredText reST also uses plain text as its method of creating test data tables This format is commonly used for documentation of Python projects and is very similar to the HTML format since the data is defined in tables in the same way as in HTML All the test data and th
52. ors to a test engineer Once created these automated tests can be repeated easily and can perform actions faster than any human person ever could Smartbear 2012 3 1 1 Benefits of automated testing The usage of some form of fast paced agile development method is very common in organizations today Test automation is becoming more of a necessity rather than an option in the software development since many upgrades are frequently made to the product under development Between the development cycles of software many tests have to be repeated often to ensure quality Every time the source code is altered these tests should be performed on the software If the software supports different operating systems and hardware configurations the amount of tests to be run increases even more Running these tests manually would be costly and time consuming on the other hand automated tests have to be created only once and after that can be run over and over again whenever needed Smartbear 2012 Some tests are nearly impossible to be run manually They might be too long or might require thousands of simultaneous users to test With test automation tests can be run simultaneously on different computers and can simulate as many virtual users interacting with the software as needed Automated testing tools are able to monitor memory contents data tables file contents and internal program states quickly and accurately Using all this gathered
53. re These applications can be anything the client needs and wants to use This will remove the need for organizations to handle the installation set up and maintenance of the required applications by themselves everything is handled by the service provider The applications are then accessible 15 through either a client interface such as a web browser or a designated program interface Webopedia 2012 Platform as a Service PaaS Platform as a service is another service model in cloud computing It is an environment that provides the possibility to develop deploy manage and integrate applications inside the cloud using the efficiency flexibility and speed offered by the cloud service SaaS is a service application that is ready to be used while PaaS is a platform allowing the client to create their own differentiating and unique applications for their exact needs IBM 2012 Infrastructure as a Service laaS By requesting Infrastructure as a Service the client has control over the software environment inside the cloud but is not required to maintain any of the equipment it is running on The client does not have to invest in their own hardware to run the software on but can request virtual machines from the laaS provider These virtual machines can then be used to contain any software the client wants and if more storage or other resources are required the service provider can easily add more of them with the help of vi
54. re precise information about the test and if the test failed it will show where and why it failed The generated log file from this test case can be seen in figure 32 48 Generated WebminLogin Test Log 20120427 17 14 36 GMT 03 00 26 minutes 36 seconds ago Test Statistics emae OO All Tests ooo 11 1 o Tn I IBI MN mmm j Statistics by Suite Total Pass Fail Graph Webminlogin 1 1 Test Execution Log E TEST SUITE WebminLogin Full Name WebminLogin Source D SmoketestsWWebminLogin txt Start End Elapsed 20120427 17 14 27 519 20120427 17 14 36 236 00 00 08 717 Status 1 critical test 1 passed 0 failed 1 test total 1 passed 0 failed a TEST CASE WebminLogin Full Name WebminLogin WebminLogin Start End Elapsed 20120427 17 14 27 651 20120427 17 14 36 235 00 00 08 584 Status PASS critical E KEYWORD SeleniumLibrary Open Browser http freenestdemoserver localdomain ff E KEYWORD SeleniumLibrary Input Text uname AdminUser E KEYWORD SeleniumLibrary Input Password password AdminUser E KEYWORD SeleniumLibrary Click Button Login E KEYWORD SeleniumLibrary Click Link Administration dont_wait E KEYWORD SeleniumLibrary Click Link Nest Server Administration E KEYWORD SeleniumLibrary Input Text user adminuser E KEYWORD SeleniumLibrary Input Password pass adminuser E KEYWORD SeleniumLibrary Click Button Login E KEYWORD SeleniumLibrary Pa
55. rmation ChangezillaAdminRights Verifies that only FreeNEST admin has access to admin tools in Changezilla Security testing correctness testing ChangezillaLogin Verifies that users logged in FreeNEST are also logged into Changezilla Correctness testing ChangezillaSearch Performs a search action in Changezilla and verifies that results are found Correctness testing CheckTabs Navigates through all the front pages of every component installed within FreeNEST Correctness testing CreateProfilePage Creates a profile page in Foswiki for a user and removes it afterwards Correctness testing CreateWikiTopic Creates a normal topic in Foswiki and removes it afterwards Correctness testing EditUser Edits user information confirms the change and restores the user to default state Correctness testing ForumAdminRights Verifies that only admins have access to administrative tools in the forum Security testing correctness testing HelpdeskAdminRights Verifies that only FreeNEST admin access to administrative tools in Helpdesk Security testing correctness testing See appendix 1 for more information IrcLogin Logs in user to IRC chat and confirms that the user can connect to the chat server Correctness testing See appendix 5 for more information TABLE 2 List of created test cases part 2 2 51
56. rojektin sis ll Ty n teoriaosassa on vertailtu automaattitestauksen ja manuaalitestauksen etuja ja rajoituksia sek esitelty eri testausk yt nt j ja testien luomisessa ja ajamisessa k ytetyt ty kalut ja ohjelmat Ty ss on my s esitelty yhden testitapauksen valmistuminen askelittain k ytt en apuna RIDE editoria joka on Robot Frameworkin oma testienluontity kalu Testien luonti onnistui ajallaan ja niiden ajaminen OpenStack pilveen luoduilla virtuaalikoneilla voitiin suorittaa Testien kattavuus kuitenkin j i tiettyjen testien suorittajaan liittyvien rajoitusten takia puutteelliseksi ja t ten kaikille FreeNEST ohjelman perustoiminnoille ei saatu viel luotua testisarjoja Avainsanat asiasanat automaatiotestaus Robot Framework RIDE FreeNEST SkyNEST OpenStack Muut tiedot dh ja pS als JYVASKYLAN AMMATTIKORKEAKOULU DESCRIPTION ay JAMK UNIVERSITY OF APPLIED SCIENCES Author s Type of publication Date OJALA Teemu Bachelor s Master s Thesis 02 05 2012 Pages Language 61 English Confidential Permission for web publication Until X Title SMOKE TESTING BENCH FOR FREENEST SERVICE Degree Programme Information Technology Tutor s LEINO Janne Assigned by RINTAM KI Marko Abstract The main objective of this study was to create a set of thirty automated tests These tests verify some of the basic functionality of different components included in Fr
57. rtualization technologies Caruso 2011 4 TECHNOLOGIES 4 1 General Many programs used to make testing easier on different aspects are being developed all the time in the web Some of them are components or complete installments that allow users to perform automated tests for different software that are in development Other programs may be used to help the testers themselves to plan new tests and keep track on tests that have already been performed for different 16 builds of the target software and even to create vast test plans for the development project All different programs and components used in this study are open source based so there are no costs using any of them as long as the products made with them are not used in any commercial way TestLink is a web based test management system that is currently included within the default installment of FreeNEST web service platform The author was already familiar with TestLink as he had used it previously therefore it was an easy choice to select it as the management part for this assembly Robot Framework is a test automation framework used to execute the desired tests Selenium RC is the component that is required by Robot Framework to understand and command web based applications like FreeNEST therefore making automated tests possible with the assembly used in this work 4 2 TestLink TestLink is a free to use web based test management program that helps its users to create
58. s of the project had to leave and the development of the FreeNEST was decelerated considerably a good opportunity to plan and design more efficient and accurate testing method was granted for the project team When the development of the FreeNEST starts speeding up again this automated test bench 28 will allow the future testers to have more time to spend on improving the testing methods and the test cases when less time has to be used on manual testing SkyNEST project is also developing and maintaining their own OpenStack cloud inside the project By setting up the automated tester inside this cloud many benefits are gained for the process of test execution such as easier and faster management of virtual instances that are needed in the testing It will also grant higher resources to perform the tests and more accurate monitoring of the performance of the instances when they are located inside a cloud Junkcloud Junkcloud is the unofficial name of the project s OpenStack cloud It is still a relatively new part of the project however it has attracted a great deal of interest from many different people outside the project Currently Junkcloud works as an laaS platform see chapter 3 3 for the test execution system developed in the FreeNEST project To enable test execution in the cloud three instances or virtual machines were created inside it These three machines are called Team Server Master Tester and Deep Forest The first
59. ster Tester Target Deep Forest nstance FreeNEST 3 Project Platform Instance Test Instance FreeNEST 1 4 Development Version Test Management for Execution Robot Manual Testing Test Automation x already installed inside F Engine Framework Testlink is alre Test Case Script Testlink is already installed inside Version n n A A A Open Stack EC2 cloud FIGURE 9 The test automation framework created inside the Junkcloud GIT Version control Repository There are still some current problems concerning this automated tester Since the Junkcloud is a very low budget cloud it is not as powerful as desired There are also some parts inside the cloud that work as a bottleneck and slow down the testing process considerably This set up is very demanding to maintain and configure compared to the tests run manually Still since this is only the first version of working test automation in this project and is still heavily under development all of the problems should be fixed in the future if wanted and many more features and improvements are most likely added to it 31 6 CREATION OF THE SMOKE TEST PLAN 6 1 Planning The main object of this thesis was to select and write a set of thirty tests that verify some basic functionality of the FreeNEST service that can then be executed automatically with the test automation framework built inside junkcloud The amount of tests to create was specified when this study was assigned
60. system The performed tests can calculate and monitor the resource usage throughput and response time of the system under test Performance testing can be executed on any applications to measure their specifications however the most usual targets are web applications since they require low latency on the website high throughput and low utilization Performance testing can be divided into load testing and stress testing Khan 2010 To ensure that a service can handle a set number of users load tests should be executed These tests can be carried out by using virtual users since they are easy to administrate Load testing is also used to observe the performance of applications in heavy load situations thus it is easier for web services to find out how many users the service can hold until the systems start failing or their performance decreases When the load placed on the system is intentionally raised above its normal amounts of usage the tests are considered as stress tests Load and stress tests can be done either manually or automatically however manual testing is not nearly as practical as automated testing since with automated testing tests can be easily repeated as many times as needed Khan 2010 13 3 2 3 Reliability testing Reliability testing is used to discover failures in the system so they can be fixed before deployment of the system in question With the help of reliability testing an estimation model is created
61. t all the test data is written in one uniform table unlike in the HTML files where the different test data tables are separated by completely different tables The test data tables in TSV files are separated by inserting one or more asterisks in the cells of the table and any text that has been written before the first asterisks will be completely ignored Figure 4 illustrates this method of dividing the tables inside a TSV file These TSV files can be created and edited with different spreadsheet programs the most common one being Microsoft Excel Robot Framework 2012 Setting Value Value Value Library OperatingSystem Variable Value Value Value T HMESSAGE Hello world Test Case Action Argument Argument My Test Documentation Example test Log S MESSAGE My Keyword tmp Another Test Should Be Equal S MESSAGE _ Hello world Keyword Action Argument Argument My Keyword Arguments path Directory Should Exist Sipath FIGURE 4 Test case in TSV format Robot Framework 2012 Plain Text Format 21 Plain Text Format uses simple txt files to create tests for Robot Framework which is the biggest benefit when comparing this format to the ones mentioned before since the text editors are very easy to use When doing tests on plain test format the separator between cells is either two or m
62. t Test Execution Execute Tests Ra rE Test Reports and Metrics G Metrics Dashboard Reguirement Specification a jan T Requirement Overview G Test Cases Assigned to Me G Search Requirements Search Requirement Specifiations Test Plan contents Assign Requirements Add Remove Platforms Generate Requirement Specification Document G Add Remove Test Cases G Update Linked Test Case Versions Test Specification Show ca tn Versions Assign Test Case Execution Test Specificatio a Jan Set Urgent Tests Search Test Cases G Assign Keywords Generate Test Specification Document Documentation Choose Document x FIGURE 2 A Screenshot of TestLink demo TestLink Projects When creating a new project with TestLink the user is asked to name the new project and specify a unique ID for it This ID is automatically added to the names of any Test Cases created within the project in question The user can also write a description for the project that can explain different aspects in slight detail such as the aim of the project or its target group There are also few settings that can be turned on for this project The most important one considering this thesis is to enable the usage of API keys Without them the project will not allow users to execute any automated tests on the project TestLink 2012 18 Test Cases After a new project has been created users can immediately start creating Test Cases for th
63. their target part of the software Unit testing is used to verify the functions of independent components of the product before they are combined with other components into a larger unit In integration testing the functionality between components is tested Units are tested in groups and their interaction between each other is monitored System testing is carried out when the full complexity of the product is assembled This is usually the most formal part of testing and it is used to ensure that the 11 product responds correctly in normal situations and handles possible exceptions in a correct way Krishna Training 2011 3 2 1 Correctness Testing The minimum requirement of any produced software is that it behaves as it should Correctness testing is used to make sure that the software works as intended and to discover and remove the incorrect behavior of the system Correctness testing is divided into three different types of testing depending on the type of person executing the tests and his her personal knowledge of the software in question These three types of testing are white box testing black box testing and their combination grey box testing Khan 2010 White box testing White box testing requires the tester to have full knowledge of the software and its source code under testing It consists of accurately driven tests where input is given to the system and then its effects are inspected inside the system process Aft
64. ther with a click of a button so if for example you wanted to create a series of test cases that will test the login instead of just one you could give ita name such as Login Tests This test will verify the login to Webmin an administrative application that is one of the applications installed with FreeNEST package You can also select the location where to save the test file and the format it will be created with The user decided to use the plain text format gt New Project ES are WebminLogin Type 9 File gt Directory Parent Directory D Smoketests Browse Format O TSV HTML FIGURE 10 Creating a new project with RIDE After the new project is created SeleniumLibrary has to be added to the used Libraries list by using the Add Library button located on the right side of the user interface while the newly created project is selected The library has to be named exactly as SeleniumLibrary for it to be recognized correctly inside the test file There are also options to insert arguments an alias and comments for the Library however they are not unnecessary for this case so they are left blank These options can be seen in figure 11 34 pes Library Name SeleniumLibrary Aras Alias Comment Give name optional arguments and optional alias of the library to import Separate multiple arguments with a pipe character like arg 1 arg 2 Alias can be used to import sam
65. ting WebminLogin Log in test for Webmin inside FreeNEST with correct user credentials Correctness testing Creation of the test case explained in chapter 6 2 WikiAdminRights Verifies that only FreeNEST admin has access to administrative tools in Foswiki Security testing correctness testing WikiNavigation Navigates trough links found in different pages of Foswiki Correctness testing WikiSearch Performs a search action in Foswiki and verifies that results are found Correctness testing See appendix 3 for more information WikiwordDashboard Confirms the Foswiki links created in front page of FreeNEST are working Correctness testing 52 The factors that prevented the creation of more perfect smoke test plan were the restrictions in the current test execution engine and my personal knowledge regarding automated testing For example creating tests that repeat endlessly in a loop would have required some more know how about programming since the tests created for robot framework can be improved by using Python program language inside the test cases and to create new test libraries In addition getting more precise and profound information about all the different ways Robot Framework is possible to create and perform tests would have required a great deal more time that simply did not have anymore In the end what managed to create was a guideline for the future autom
66. ually classified as button therefore using the keyword Click Button cannot be used this time Instead another one called 46 Click Element has to be used which works very much like the Click Button keyword The ID of the button can be seen in the source code and it can be added as the locator for the keyword The last keyword to add into the test is Close Browser This is not unnecessary to do but running hundreds of tests and not closing any of the browsers will cause complications on the machine doing the test such as decrease in performance The finished test case can be seen in figure 30 Sa ENS L RIDE WebminLogin xA File Edit Tools Navigate Run Help lt OBIE a WebminLogin Edit Run amp WebminLogin r y 5 aay External Resources WebminLogin Open Browser http freenestdemoserver localdo ff jad Input Text uname AdminUser Input Password password AdminUser Click Button Login Click Link Administration dont wait Click Link Nest Server Administration Input Text user adminuser Input Password pass adminuser Click Button Login 0 0 S a vu bs WN Page Should Contain Webmin version Click Element logoutBtn Close Browser Open file containing tests FIGURE 30 Completed test case in RIDE Now that the test is completed it can be executed to make sure there are no errors found in the steps It is also recommendable to run the test after add
67. www webhostingunleashed com features cloud computing benefits Webopedia n d Saas Software as a Service Storage as a Service Referred on April 25 2012 http www webopedia com TERM S SaaS html Wikipedia n d Cloud Computing Referred on April 28 2012 http en wikipedia org wiki Cloud computin 56 APPENDIX 1 TESTCASE THAT VERIFIES ADMIN RIGHTS ON HELPDESK Pa This test case will verify that only user with administrator rights will have access to the administrative tools in Helpdesk AdminUser should be automatically recognized as administrator in Helpdesk immediately after logging into FreeNEST and DemoUser should have access only to the normal users section AER Settings Library SeleniumL xxx Test Cases HelpdeskAdminRights Open Browser http ick Button Login ick Link Helpdesk ick Link Setup Close browser ibrary freenestdemoserver localdomain ff Input Text uname AdminUser Input Password password AdminUser age Should Contain Welcome AdminUser Admin age Should Contain Select the category to configure ick Element logoutBtn Open Browser http freenestdemoserver localdomain ff Click Button Login Click Link Helpdesk Page Should Contain Input Text uname DemoUser Input Password password DemoUser Describe the Problem Action Page Should Not Contain Welcome Dem
68. xt editor but working without other tools is not recommended The author used RIDE editor to create the test case files and saved them in txt format Additionally the help of Firefox plug in called firebug was used which makes it much easier to find certain identifiers from the source code of the web application To verify that the commands are correctly written in the test file Robot Framework was used to execute the tests after every set of several commands To use the keywords from SeleniumLibrary Selenium Server has to be running or the test will not work Robot Framework uses Python Jython and IronPython programming languages and requires atleast one of them installed on the operating system The one selected to 33 execute the tests here is Python All these tools were used on Windows 7 operating system To start creating the test file RIDE has to be launched first This is done by writing ride py into the Windows command console The instructions are given below as follows The user is instructed to create a new project and it can be simply selected from the dropdown list under file or by pressing key combination ctrl N This will cause a new window presented in figure 10 to open User is then asked to give a name for the project Since the author is making a project that will just have one test in it it is named after the test One project can contain multiple test cases and they can all be run one after ano
69. ypertext markup language HTML tab separated values TSV plain text or reStructuredText rest file format All of these are somewhat similar to each other but have some minor differences and benefits over each other Robot Framework 2012 HTML Format Recognized extensions of files for test cases created in HTML format are html htm and xhtml In these files all the test data is written inside many separate text tables as shown in figure 3 The first cell in each test data table is used by Robot Framework to identify different types of tables inside the test file If it is not one of the recognized tables it will be ignored completely The tables can be created using any kind of editor however using a graphical editor is more favorable as you can see the tables while you edit them Robot Framework 2012 20 Setting Value Value Value Library OperatingSystem Variable Value Value Value S MESSAGE Hello world Test Case Action Argument Argument My Test Documentation Example test Log SIMESSAGE My Keyword tmp Another Test Should Be Equal S MESSAGE Hello world Keyword Action i Argument Argument My Keyword Arguments path Directory Should Exist path FIGURE 3 Test case in HTML format Robot Framework 2012 TSV Format TSV files are edited very much like files in HTML format The main difference is tha

SMOKE TESTING BENCH FOR FREENEST SERVICE

Contents

Download Pdf Manuals

Related Search

Related Contents