Home

Final report - Chair of Software Engineering

1. C O 79 Test Wizard Master Thesis BOOLEAN REF to integer passed 10 0 0 0 0 BOOLEAN REF set item passed 10 0 0 0 0 BOOLEAN REF infix and passed 7 3 0 0 0 BOOLEAN REF infix and then passed 3 7 0 0 0 BOOLEAN REF infix implies passed 8 2 0 0 0 BOOLEAN REF prefix not passed 10 0 0 0 0 BOOLEAN REF infix or passed 6 4 0 0 0 BOOLEAN REF infix or else passed 8 2 0 0 0 BOOLEAN REF infix xor passed 8 2 0 0 0 BOOLEAN REF out passed 10 0 0 0 0 BOOLEAN REF generator passed 10 0 0 0 0 BOOLEAN REF generating type passed 10 0 0 0 0 BOOLEAN REF conforms to passed 10 0 0 0 0 BOOLEAN REF same type passed 10 0 0 0 0 BOOLEAN REF consistent passed 10 0 0 0 0 BOOLEAN REF is equal passed 4 6 0 0 0 BOOLEAN REF standard is equal passed 6 4 0 0 0 BOOLEAN REF equal failed 0 0 0 0 0 BOOLEAN REF standard equal failed 5 0 0 0 5 BOOLEAN REF deep equal failed 6 0 0 0 4 BOOLEAN REF passed 9 1 0 0 0 BOOLEAN REF standard copy passed 6 4 0 0 0 BOOLEAN R
2. put integer no call was valid 0 10 0 0 0 TUPLE put integer 32 no call was valid 0 10 0 0 0 TUPLE put integer 8 no call was valid 0 10 0 0 0 TUPLE put integer 16 no call was valid 0 10 0 0 0 TUPLE put integer 64 no call was valid 0 10 0 0 0 TUPLE is boolean item no call was valid 0 10 0 0 0 TUPLE is character item no call was valid 0 10 0 0 0 TUPLE is wide character item no call was valid 0 10 0 0 0 TUPLE double item no call was valid 0 10 0 0 0 TUPLE is integer 8 item no call was valid 0 10 0 0 0 TUPLE is integer 16 item no call was valid 0 10 0 0 0 TUPLE is integer item no call was valid 0 10 0 0 0 15 integer 32 item no call was valid 0 10 0 0 0 TUPLE is integer 64 item no call was valid 0 10 0 0 0 TUPLE j is pointer item no call was valid 0 10 0 0 0 TUPLE is real item no call was valid 0 10 0 0 0 TUPLE is reference item no call was valid 0 10 0 0 0 TUPLE is numeric item no call was valid 0 10 0 0 0 TUPLE is uniform passed 10 0 0 0 0 TUPLE is uniform boolean passed 10 0 0 0 0 TUPLE is uniform character passed 10 0 0 0 0 TUPLE is uniform wide character passed 10 0 0 0 0 TUPLE is uniform double passed 10 0 0 0 0 TUPLE is uniform integer 8 passed 10 0 0 0 0 TUPLE uniform integer 16 passed 10 0 0 0 0 TUPLE is uniform intege
3. INTEGER 8 infix V failed 4 1 5 0 0 INTEGER 8 infix failed 6 0 4 0 0 INTEGER 8 REF infix failed 5 0 5 0 0 INTEGER 8 boolean failed 7 0 3 0 0 INTEGER 8 integer 8 failed 7 0 3 0 0 INTEGER 8 integer 16 failed 5 0 5 0 0 INTEGER 8 REF to integer failed 6 0 4 0 0 INTEGER 8 integer 32 failed 8 0 2 0 0 INTEGER 8 REF integer 64 failed 6 0 4 0 0 INTEGER 8 REF to hex string failed 7 0 3 0 0 INTEGER 8 hex character failed 7 1 2 0 0 INTEGER 8 REF to character failed 6 3 1 0 0 INTEGER 8 infix amp failed 4 1 5 0 0 INTEGER 8 REF bit and failed 4 1 5 0 0 INTEGER 8 infix failed 6 0 4 0 0 INTEGER 8 REF or failed 8 0 2 0 0 INTEGER 8 REF xor failed 3 0 7 0 0 INTEGER 8 REF not failed 7 0 3 0 0 INTEGER 8 REF shift failed 1 6 3 0 0 INTEGER 8 REF infix lt lt failed 2 7 1 0 0 INTEGER 8 REF shift left failed 1 7 2 0 0 INTEGER 8 REF gt gt failed 2 3 5 0 0 INTEGER 8 REF shift right failed 2 4 4 0 0 INTEGER 8 REF test failed 0 8 2 0 0 INTEGER 8 REF set bit failed 0 3 7 0 0 INTEGER 8 set bit with mask failed 6 0 4 0 0 INTEGER 8 failed 9 0 1 0 0 INTEGER 8 generator failed 3 0 7 0 0 INTEGER 8 REF generat
4. STRING tail passed 8 2 0 0 0 STRING keep tail passed 6 4 0 0 0 STRING adjust passed 10 0 0 0 0 STRING right adjust passed 10 0 0 0 0 STRING share passed 9 1 0 0 0 STRING put passed 6 4 0 0 0 STRING precede passed 10 0 0 0 0 STRING prepend character passed 10 0 0 0 0 STRING prepend failed 8 1 1 0 0 STRING prepend boolean passed 10 0 0 0 0 STRING prepend double passed 10 0 0 0 0 STRING prepend integer passed 10 0 0 0 0 STRING prepend real passed 10 0 0 0 0 STRING prepend string failed 6 0 4 0 0 STRING append passed 8 2 0 0 0 STRING infix passed 9 1 0 0 0 STRING string passed 10 0 0 0 0 STRING append integer passed 10 0 0 0 0 STRING append real passed 10 0 0 0 0 STRING double passed 10 0 0 0 0 STRING append character passed 10 0 0 0 0 STRING extend passed 10 0 0 0 0 STRING append boolean passed 10 0 0 0 0 STRING insert passed 6 4 0 0 0 STRING insert string failed 3 4 3 0 0 STRING character passed 8 2 0 0 0 STRING remove passed 6 4 0 0 0 STRING remove head passed 8 2 0 0 0 STRING remove substring passed 1 9 0 0 0 STRING tail passed 8 2 0 0 0 STRING prune passed 10 0 0 0 0 STRING passed 10 0 0 0 0 STRING prune all leading passed 10 0 0 0 0 STRING all trailing passed 10 0 0 0 0
5. REAL REF infix gt passed 10 REAL REF max passed 9 REAL REF min passed 10 CHARACTER REF item passed 10 CHARACTER REF code passed 10 CHARACTER REF hash code passed 10 CHARACTER REF Min value passed 10 CHARACTER REF Max value passed 10 CHARACTER REF is hashable passed 10 CHARACTER REF infix lt passed 10 CHARACTER_REF is_equal passed 7 CHARACTER_REF three_way_comparison passed 9 CHARACTER_REF infix passed 3 CHARACTER_REF infix passed 5 CHARACTER_REF infix passed 10 CHARACTER_REF next passed 10 CHARACTER_REF previous passed 10 CHARACTER_REF set_item passed 10 CHARACTER_REF out passed 10 CHARACTER_REF as_upper passed 10 CHARACTER REF upper passed 10 CHARACTER REF as lower passed 10 CHARACTER REF lower passed 10 CHARACTER REF is lower passed 10 CHARACTER REF is upper passed 10 CHARACTER REF digit passed 10 CHARACTER REF is alpha passed 10 CHARACTER REF infix lt passed 10 CHARACTER REF infix gt passed 10 CHARACTER REF infix gt passed 9 CHARACTER REF max passed 9 CHARACTER REF min passed 9 CHARACTER REF generator passed 10 CHARACTER REF generating type passed 10 CHARACTER REF conforms to passed 8 CHARACTER REF same type passed 10 CHARACTER REF consistent passed 10 CHARACTER
6. 9 8 6 9 0 8 9 9 9 5 0 0 5 5 6 6 INTEGER 16 REF divisible failed 5 INTEGER 16 REF exponentiable failed 5 INTEGER 16 REF is hashable failed 7 INTEGER 16 REF is valid character code failed 7 INTEGER 16 REF abs failed 5 INTEGER 16 REF infix failed 7 INTEGER 16 REF infix failed 4 INTEGER 16 REF infix failed 3 INTEGER 16 REF infix failed 2 INTEGER 16 REF prefix failed 7 INTEGER 16 REF prefix failed 7 INTEGER 16 REF infix failed 1 INTEGER 16 REF infix V failed 0 INTEGER 16 REF infix failed 8 INTEGER 16 REF infix failed 6 INTEGER 16 REF boolean failed 9 INTEGER 16 REF integer 8 failed 1 INTEGER 16 REF to integer failed 9 INTEGER 16 REF to integer 32 failed 8 INTEGER 16 REF to integer 16 passed 10 INTEGER 16 REF integer 64 passed 10 INTEGER 16 REF to hex strin failed 7 INTEGER 16 REF to hex character failed 2 INTEGER 16 REF to character failed 5 INTEGER 16 REF infix amp failed 4 INTEGER 16 REF bit and failed 4 INTEGER 16 REF infix failed 3 INTEGER 16 REF bit or failed 2 INTEGER 16 REF xor failed 4 INTEGER 16 REF bit not failed 3 INTEGER 16 REF bit shift failed 4 INTEGER 16 REF infix lt lt failed 0 INTEGER 16 REF bit shift left failed 2 DO HA A ro CO COo O O O O OJ O O O O ro W O OM TO
7. no precondition assertion other possible class feature result exception violation violation exception catcall ANY generator passed 10 0 ANY generating type passed 10 0 ANY conforms to passed 10 0 ANY same type passed 10 0 ANY consistent failed 2 0 ANY is equal failed 0 2 ANY standard is equal failed 1 0 ANY equal failed 2 0 ANY standard equal failed 0 0 10 deep equal failed 2 0 ANY failed 0 3 ANY standard copy failed 1 0 ANY clone passed 10 0 ANY standard clone passed 10 0 ANY standard twin passed 10 0 ANY deep clone passed 10 0 1 0 ANY deep co failed 1 ANY setup failed 3 ANY io passed 10 0 ANY out passed 10 0 ANY tagged out passed 10 0 ANY print passed 10 0 ANY Operating environment passed 10 0 ANY default rescue passed 10 0 ANY default create passed 10 0 ANY do nothing passed 10 0 ANY default passed 10 0 ANY default pointer passed 10 0 ANY Void passed 10 0 TUPLE make passed 10 0 TUPLE item no call was valid 0 10 TUPLE infix no call was valid 0 10 no was valid 0 10 TUPLE boolean item no call was valid 0 10 TUPLE character item no call was valid 0 10 TUPLE double item no call was valid 0 10 TUPLE integer 8 item no call was valid 0 10 TUPLE integer 1
8. PI OD 74 COMER PE DS 65 7 1 1 Creation procedures and modifiers 65 quls E ban eec VE E bera 66 8 lt veris eid OF 8 1 TR 67 542 Commands anes 68 8 3 Launch generated 72 9 HA LO 9 1 Cluster and classes of the library 2 75 OD SPEStTESUIES iocos 77 9 2 1 The reference types of the base types 77 9 2 2 Cluster kernel Classic 86 10 TE vii Test Wizard Master Thesis LIST OF TABLES Table 1 Planned schedule and actual schedule 13 Table 2 Predefined default bounds for base types essere 18 iU DEED 35 Tabled Sorting ot e s oy en 36 Table 5 Container and objects 38 Table 6 Default initialization 01 21 60000001000 40 Table 7 Assertion levels ertet rne Ro eere ete eo boves eoa des ede 50 Table 8 The file details generated tes
9. 87 Test Wizard Master Thesis TUPLE passed 10 0 0 0 0 TUPLE tagged out passed 10 0 0 0 0 TUPLE print passed 10 0 0 0 0 TUPLE Operating environment passed 10 0 0 0 0 TUPLE default rescue passed 10 0 0 0 0 TUPLE default create passed 10 0 0 0 0 TUPLE do nothing passed 10 0 0 0 0 TUPLE default passed 10 0 0 0 0 TUPLE default pointer passed 10 0 0 0 0 TUPLE Void passed 10 0 0 0 0 TUPLE mismatch information passed 10 0 0 0 0 STRING make passed 8 2 0 0 0 STRING make empty passed 10 0 0 0 0 STRING make filled passed 9 1 0 0 0 STRING make from string passed 8 2 0 0 0 STRING from c no call was valid 0 10 0 0 0 STRING from_c no call was valid 0 10 0 0 0 STRING from c substring no call was valid 0 10 0 0 0 STRING adapt passed 10 0 0 0 0 STRING remake passed 5 5 0 0 0 STRING 2 8 0 0 0 STRING infix passed 2 8 0 0 0 STRING code passed 2 8 0 0 0 STRING code passed 10 0 0 0 0 STRING False constant passed 10 0 0 0 0 STRING True constant passed 10 0 0 0 0 STRING shared with passed 10 0 0 0 0 STRING index of passed 5 5 0 0 0 STRING 1 index of passed 5 5 0 0 0 STRING substring index in bounds no call was valid 0 1
10. neat 70 Figure 33 The file bounds of the demo application 71 Figure 34 Features to enable and disable assertion checking 71 Figure 35 Extract of the test results of testing the class CHARACTER REF in XML EE 74 1X Test Wizard Master Thesis 1 INTRODUCTION Testing that a program satisfies its requirements is necessary to prove to the final consumer that the product works However testing is often regarded as an expensive and non rewarding activity Most companies just produce good enough software and ship products that still contain known bugs These bugs are often expensive to correct and appear rarely Software products are becoming more complex interconnecting more and more intricate technologies across multiple operating environments The size of software products is no longer measured in terms of thousands of lines of code but millions of lines of code This increasing complexity of many software products along with a decreasing average market life expectancy has heightened concerns about software quality The purpose of software engineering is to build high quality software hence the importance of testing Testing is a crucial part of software engineering The cost of testing can range from 50 to 75 percent of the software development budget 13 Software non performance and failure are expensive The media are full of reports of the catastrophic impact of softw
11. esse 42 Figure 16 Second code piece of feature search argument 42 Figure 17 Generated feature create integer bounds eee 43 Figure 18 Generated deferred class TW 204220244042 ee teet esterne ta epoca 47 Figure 19 Generated class TW TESTI MAX for feature test 1 5 48 Figure 20 Generation Of test_feature aue eed ae natn Fa eb e pa ERR 49 Fig re 21 Catcall onions o bred 50 Figure 22 Feature copy of class ANY on expanded types 2 53 Figure 23 Power operation errors aoi asset 54 Test Wizard Master Thesis Figure 24 Code extract of feature character justify of class STRING 56 Figure 25 Object THiodITIC 1 60 Figure 26 Constrained generic o diee ie e ura ies 61 anata Ste Gath deal Sus eth 62 Figure 28 Feature analyze universe of class TW TEST WIZARD 68 Figure 29 Features to set the scope of the test eco nee reri s eue eet 69 Figure 30 Features to retrieve clusters classes and features of the library under test 69 Figure 31 Features to set the exhaustiveness d tae ee 70 Figure 32 Features to set Cc oe siete
12. 11 Test Wizard Master Thesis DEADLINE Milestone Deadline 1 2003 08 15 M2 2003 09 05 M3 2003 09 05 MA 2003 09 19 M5 2003 10 10 M6 2003 10 24 M7 2003 11 07 M8 2003 11 14 M9 2003 11 21 M10 2003 11 28 M11 2003 12 05 M12 2003 12 19 M13 2004 01 12 M14 2004 01 12 M15 2004 01 12 TENTATIVE SCHEDULE tlala S fa fa 7 amp 5 ja fs S Task amp G 31 32 33 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 Master Project 30 56 Presentation P1 Intermediary presentation 48148 P2 Final presentation 58 58 Design M1 Software architecture 30 33 Implementation M2 Eiffel analyzer 34136 M3 Information handler 36 36 4 Code Generator 37 38 5 Context handler amp context generator 39 41 M6 Call simulator 42143 7 Test result handler 44145 8 Demo application 46 46 M10 Database storing 48148 M11 Database query handler 49149 M12 Test scenario handler 50 51 Documentation M9 Intermediary report 44 48 M13 Developer manual 49 56 M14 User manual 51 56 M15 Thesis report 44 56 12 2 Feb 2004 e Test Wizard Master Thesis Task Master Project Presentation P1 Intermediary presentation
13. passed 6 4 0 0 0 INTEGER REF prefix passed 10 0 0 0 0 INTEGER REF prefix passed 10 0 0 0 0 INTEGER REF infix passed 9 1 0 0 0 80 Test Wizard Master Thesis INTEGER REF infix V passed 6 4 0 0 0 INTEGER REF infix passed 10 0 0 0 0 INTEGER REF infix passed 10 0 0 0 0 INTEGER_REF to_boolean passed 10 0 0 0 0 INTEGER_REF to_integer_8 passed 4 6 0 0 0 INTEGER_REF to_integer_16 passed 9 1 0 0 0 INTEGER_REF to_integer passed 10 0 0 0 0 INTEGER_REF to_integer_32 passed 10 0 0 0 0 INTEGER_REF to_integer_64 passed 10 0 0 0 0 INTEGER_REF to_hex_string passed 10 0 0 0 0 INTEGER_REF to_hex_character passed 1 9 0 0 0 INTEGER_REF to_character passed 5 5 0 0 0 INTEGER_REF infix amp passed 8 2 0 0 0 INTEGER_REF bit_and passed 9 1 0 0 0 INTEGER REF infix passed 9 1 0 0 0 INTEGER REF bit or passed 8 2 0 0 0 INTEGER REF bit xor passed 9 1 0 0 0 INTEGER REF bit not passed 10 0 0 0 0 INTEGER REF bit shift passed 6 4 0 0 0 INTEGER REF infix lt lt passed 6 4 0 0 0 INTEGER_REF bit_shift_left passed 3 7 0 0 0 INTEGER REF infix gt gt passed 3 7 0 0 0 INTEGER REF bit shift right passed 1 9 0 0 0 INTEGER REF bit test passed 2 8 0 0 0 INTEGER REF set bit no call was valid 0 10 0 0 0 INTEGER REF Set bit with
14. O O O O O AIN Aw OC O PO CO BY PO OT S TO CO CO BH CO RYO OO O H 83 Test Wizard Master Thesis INTEGER 16 REF infix gt gt failed INTEGER 16 REF bit shift right failed INTEGER 16 REF bit test failed INTEGER 16 REF set bit failed INTEGER 16 REF set bit with mask failed INTEGER 16 REF out failed INTEGER 16 REF generator failed INTEGER 16 REF generating type failed INTEGER 16 REF conforms to failed INTEGER 16 REF same type failed INTEGER 16 REF consistent failed INTEGER 16 REF standard is equal failed INTEGER 16 REF equal failed INTEGER 16 REF standard equal failed INTEGER 16 REF deep equal failed INTEGER 16 REF copy failed INTEGER 16 REF standard copy fa
15. P2 Final presentation Design M1 Software architecture Implementation M2 Eiffel analyzer M3 Information handler M4 Code Generator M5 Context handler amp context generator M6 Call simulator M7 Test result handler M8 Demo application M10 Database storing M11 Database query handler M12 Test scenario handler Documentation M9 Intermediary report M13 Developer manual M14 User manual M15 Thesis report 44 49 38 51 55 44 52 ENS Actual schedule 3 DESIGN OF THE TEST WIZARD The goal of this master thesis was to develop the tool Test Wizard which generates test cases automatically from the assertions expressed in Eiffel libraries As mentioned before the GUI development was not part of this project Furthermore all the database handling as well as the test scenario handler is not implemented yet The project plan schedule and actual schedule are compared in the following table 21 Jul 2003 28 Jul 2003 44 Aug 2003 11 Aug 2003 30 31 32 33 S End 58 33 33 36 39 36 37 38 40 41 51 43 51 45 55 46 55 48 49 51 48 56 51 56 55 56 56 6 8 34 35 36 37 6 Okt 2003 22 Sep 2003 29 Sep 2003 39 40 41 13 Okt 2003 0 Okt 2003 27 Okt 2003 3 Nov 2003 10 Nov 2003 17 Nov 2003 24 Nov 2003 1 Dez 2003 8 Dez 2003 p 42 43 44 45 46 47 48 49 50 Table 1 Planned schedule and actual schedule
16. REAL and DOUBLE The features generate number creation generate boolean creation and generate character creation fill the container with predefined default bounds see Table 2 The features generate creation and generate deferred creation create the remaining features create xxx object Furthermore the feature generate any creation creates the feature create object of the class TW CONTEXT Finally the feature generate generates the code of feature create any descendants It takes an object of every container and adds it to the container of ANY if the object is not Void or a default value The third part of the class TW CONTEXT is the modification The command generate all modification called by the feature generate context class generates this part the generated features are named xxx object The command relies on only two features generate modification and generate deferred modification The first feature creates the code for effective classes and the second for deferred classes The modifiers extracted with the help of the Eiffel analyzer are needed They can be accessed by the attribute modifiers of class OBJECT Both features loop through the modifiers and handle them provided that the modifier is not a feature of class ANY and is not using a pointer To check these properties the queries is any feature and is using pointer are disposal features generate prefix feature generate
17. Reusable Software The Base Object Oriented Component Libraries Prentice Hall 1994 Bertrand Meyer Object Oriented Software Construction 2nd edition Prentice Hall 1997 Bertrand Meyer Design by Contract Prentice Hall in preparation Robert Mitchell and Jim McKim Design by Contract by example Addison Wesley 2002 Eiffel Software Inc EiffelBase Retrieved June 2003 from http docs eiffel com libraries base index html National Institute of Standards and Technology NIST The Economic Impacts of Inadequate Infrastructure for Software Testing Planning Report 02 3 RTI Project Number 7007 011 May 2002 http www nist gov director prog ofc report02 3 pdf 91
18. STRING wipe out passed 10 0 0 0 0 STRING clear_all passed 10 0 0 0 0 STRING adapt size passed 10 0 0 0 0 STRING resize passed 8 2 0 0 0 STRING grow passed 10 0 0 0 0 STRING lower passed 10 0 0 0 0 STRING upper passed 10 0 0 0 0 STRING _ left justify passed 10 0 0 0 0 STRING center justify could not be tested 0 0 0 0 0 STRING right justify passed 10 0 0 0 0 STRING character justify could not be tested 0 0 0 0 0 STRING 10 lower passed 10 0 0 0 0 STRING 10 upper passed 10 0 0 0 0 STRING _ integer no was valid 0 10 0 0 0 STRING integer 64 no call was valid 0 10 0 0 0 STRING real no call was valid 0 10 0 0 0 STRING to double no call was valid 0 10 0 0 0 STRING to boolean no call was valid 0 10 0 0 0 STRING linear representation passed 10 0 0 0 0 STRING split passed 10 0 0 0 0 STRING to c passed 10 0 0 0 0 STRING mirrored passed 10 0 0 0 0 STRING mirror passed 10 0 0 0 0 STRING substring passed 10 0 0 0 0 STRING multiply passed 7 3 0 0 0 STRING out passed 10 0 0 0 0 STRING inserted passed 10 0 0 0 0 STRING fill passed 10 0 0 0 0 89 Test Wizard Master Thesis STRING is empty passed 10 0 0 0 0 STRING empty passed 10 0 0 0 0 STRING object comparison passed 1
19. TEST GENERATOR are generate test class and generate all classes which relies on generate feature class The first feature generates the file tw test e that contains the deferred class TW TEST see Figure 18 the others generate a file respectively a class for each feature under test These classes are all descendants of class TW TEST files are placed in the directory tw test The place of the files is important to generate the ace file indexing description Test Wizard Automatic generated file containing the ancestor of all Test classes date Date 2004 01 20 21 33 54 deferred class TW TEST inherit EXCEPTIONS feature Run run a context TW CONTEXT a file PLAIN TEXT FILE is Call test feature call in file in context a context Set exceptions local failure INTEGER do if failure 1 then test feature a context end rescue a file put string associated class name a file put string a file put string feature name a file put string if original exception precondition then if tag void and tag name has substring test wizard then set void target a file put string void target object else set precondition violation exception a file put string precondition violation end elseif assertion violation then if tag name void and tag name has substring test wizard then set void target a file put string void target object else set assertion violat
20. working directory are used to switch between working directories For compiling and running the feature system is used it passes a request to the operating system to execute the argument of type STRING Two directories are created for the generated files the test directory and the code directory The feature create directory either creates a directory or deletes the contents of the directory if it already exists The directory as well as the directory name access can be found under the feature clause File access The test directory contains all the generated files for the context and the feature tests the code directory contains the ace file and the root class The access to the test criteria fixed by the information handler is also stored in the class TW CODE GENERATOR SUPPORT The attribute fest criteria offers access to all attributes of the information handler whereas the scope the context and the system information are stored in the separate attributes scope context and structure Furthermore the class offers six features see Figure 9 to generate Eiffel code used in every file like the indexing part Each of these six features needs as argument the file that should be generated The feature generate indexing needs besides the description as a string generate class has as argument a boolean to know if a deferred class will be generated and of course the class name given as a string generate inherit has as second argument the class
21. 8 2 0 0 0 INTEGER 64 REF to integer passed 8 2 0 0 0 INTEGER 64 REF to integer 32 passed 8 2 0 0 0 INTEGER 64 REF integer 64 passed 10 0 0 0 0 INTEGER 64 REF to hex string passed 10 0 0 0 0 INTEGER 64 REF to hex character passed 4 6 0 0 0 INTEGER 64 REF to character passed 5 5 0 0 0 INTEGER 64 REF infix amp passed 7 3 0 0 0 INTEGER 64 REF bit and passed 8 2 0 0 0 INTEGER 64 REF infix passed 9 1 0 0 0 INTEGER 64 REF bit or passed 9 1 0 0 0 INTEGER 64 REF bit xor passed 6 4 0 0 0 INTEGER 64 REF bit not passed 10 0 0 0 0 INTEGER 64 REF shift passed 6 4 0 0 0 INTEGER 64 REF infix lt lt passed 3 7 0 0 0 INTEGER 64 REF bit shift left passed 4 6 0 0 0 INTEGER 64 REF infix gt gt passed 4 6 0 0 0 INTEGER 64 REF bit shift right passed 2 8 0 0 0 INTEGER 64 REF bit test passed 2 8 0 0 0 INTEGER 64 REF set bit passed 3 7 0 0 0 INTEGER 64 REF bit with mask passed 10 0 0 0 0 INTEGER 64 REF out passed 10 0 0 0 0 INTEGER 64 REF generator passed 10 0 0 0 0 INTEGER 64 REF generating type passed 10 0 0 0 0 INTEGER 64 REF conforms to passed 10 0 0 0 0 INTEGER 64 REF same type assed 9 1 0 0 0 INTEGER 64 REF consistent passed 10 0 0 0 0 INTEGER 64 REF standard is equal passed 10 0 0 0 0 INTEGER 64 REF equal failed 5 0 0 0 5 INTEGER 64 REF standard equal failed 1 0 0 0 9 INTEGER 64 REF deep equal failed 3 0 0 0 7 INTEGER 64 REF copy passed 9 1 0 0 0 INTEGER 64 REF standar
22. CHARACTER hash code passed 10 0 0 0 0 0 CHARACTER REF Min value passed 10 0 0 0 0 0 CHARACTER REF Max value passed 10 0 0 0 0 0 CHARACTER REF is hashable passed 10 0 0 0 0 0 CHARACTER infix passed 10 0 0 0 0 0 CHARACTER is equal passed 8 2 0 0 0 0 CHARACTER REF three way comparison passed 9 1 0 0 0 0 CHARACTER infix passed 5 5 0 0 0 0 CHARACTER infix passed 6 4 0 0 0 0 CHARACTER infix passed 10 0 0 0 0 0 CHARACTER_REF next passed 10 0 0 0 0 0 CHARACTER_REF previous passed 10 0 0 0 0 0 CHARACTER_REF set_item passed 10 0 0 0 0 0 CHARACTER_REF out passed 10 0 0 0 0 0 CHARACTER_REF as_upper passed 10 0 0 0 0 0 CHARACTER_REF upper passed 10 0 0 0 0 0 CHARACTER_REF as_lower passed 10 0 0 0 0 0 CHARACTER_REF lower passed 10 0 0 0 0 0 CHARACTER_REF is_lower passed 10 0 0 0 0 0 CHARACTER_REF is upper passed 10 0 0 0 0 0 CHARACTER is digit passed 10 0 0 0 0 0 CHARACTER REF is alpha passed 10 0 0 0 0 0 CHARACTER REF infix lt passed 10 0 0 0 0 0 CHARACTER REF infix gt passed 8 2 0 0 0 0 CHARACTER REF infix gt passed 10 0 0 0 0 0 CHARACTER REF max passed 9 1 0 0 0 0 CHARACTER REF min passed 7 3 0 0 0 0 CHARACTER generator passed 10 0 0 0 0 0 CHARACTER REF generating type passed 10 0 0 0 0 0 CHARACTER REF conforms to passed 9 1 0 0 0 0 CHARACTER REF same type passed 9 1 0 0 0 0 CHARACTER REF consistent passed 10 0 0 0 0 0 CHARACTER REF standard is equal passed 10 0 0 0
23. Number of calls with precondition violations assertion violation exception INTEGER Number of calls with assertion violations other exception INTEGER Number of calls with other exceptions possible INTEGER Number of calls with possible catcalls void target INTEGER Number of calls with void target object feature Exceptions setting end set no exception is Increment no exception do no exception no exception 1 end set precondition violation exception is Increment precondition violation exception do precondition violation exception precondition violation exception 1 end set assertion violation exception is Increment assertion violation exception do assertion violation exception assertion violation exception 4 1 end set other exception is Increment other exception do other exception other exception 1 end set possible is Increment possible do possible catcall possible catcall 1 end set void targetis Increment void target do void target void target 1 end Figure 18 Generated deferred class TW TEST 47 Test Wizard Master Thesis The feature generate all classes loops through all features in the scope and calls generate feature class for each feature The scope is a test criteria fixed by the information handler It generates a class for every feature The class consists only of two features a creation procedure an
24. Project plan schedule 15 Dez 2003 51 22 Dez 2003 29 Dez 2003 5 Jan 2004 12 Jan 2004 19 Jan 2004 26 Jan 2004 52 53 54 55 56 57 13 2 Feb 2004 a 8 Test Wizard Master Thesis This chapter describes the actual design of the Test Wizard and explains the difference from the project description Figure 2 shows the final architecture of the Test Wizard Test Wizard now follows a four step process Eiffel analyzer Parsing the contract equipped Eiffel library provided as input to get the system information Information handler Gathering user information to define the test scenario Code generator Generating and running the corresponding test executable Displaying the results to the user 1 2 3 4 5 6 7 8 14 Input Output L 1 Storage Figure 2 Actual architecture of the Test Wizard Gather system information Display system information to the user and gather user information Build the test scenario from the criteria selected by the user Adapt test scenario Store the test scenario into a database for regression testing Generate a test executable corresponding to the test scenario Run the executable it creates a pool of objects the Context and performs feature calls on the pool of instantiated objects Output the results to the users generate XML files and Excel compatible text files Probably store information about the test executa
25. REF standard is equal passed 10 CHARACTER REF equal failed 1 CHARACTER REF standard equal failed 6 CHARACTER REF deep equal failed 2 CHARACTER REF passed 9 CHARACTER REF standard copy passed 10 CHARACTER REF clone passed 10 CHARACTER REF standard clone passed 10 CHARACTER REF standard twin passed 10 CHARACTER REF deep clone passed 10 CHARACTER REF deep copy passed 10 CHARACTER REF setup passed 10 CHARACTER REF io passed 10 CHARACTER REF tagged out assed 10 CHARACTER REF print passed 10 CHARACTER REF Operating environment passed 10 CHARACTER REF default rescue passed 10 CHARACTER REF default create passed 10 CHARACTER REF do nothing passed 10 CHARACTER REF default passed 10 CHARACTER REF default pointer passed 10 CHARACTER REF Void passed 10 BOOLEAN REF item passed 10 BOOLEAN REF hash code passed 10 BOOLEAN REF is hashable passed 10 o oj oiojojojoioj oj oj oj oj ojoj o ojojoj o oj2 oj oj ojoloj oj imo o 2 23 oj oloo jojoj o oj ojo oj o ojo oj on 2 o ojo o ojoooojoli io
26. REF divisible passed 9 1 0 0 0 DOUBLE REF exponentiable passed 7 3 0 0 0 DOUBLE REF is hashable passed 10 0 0 0 0 DOUBLE REF truncated to integer passed 10 0 0 0 0 DOUBLE REF truncated to integer 64 passed 10 0 0 0 0 DOUBLE REF truncated to real passed 10 0 0 0 0 DOUBLE REF ceiling passed 10 0 0 0 0 DOUBLE REF floor passed 10 0 0 0 0 DOUBLE REF rounded passed 10 0 0 0 0 DOUBLE REF abs passed 10 0 0 0 0 DOUBLE REF infix passed i 3 0 0 0 DOUBLE_REF infix passed 10 0 0 0 0 DOUBLE_REF infix passed 9 1 0 0 0 DOUBLE_REF infix passed 9 1 0 0 0 DOUBLE REF infix assed 10 0 0 0 0 DOUBLE REF prefix passed 10 0 0 0 0 DOUBLE REF prefix passed 10 0 0 0 0 DOUBLE REF out passed 10 0 0 0 0 DOUBLE REF generator passed 10 0 0 0 0 DOUBLE REF generating type passed 10 0 0 0 0 DOUBLE REF conforms to passed 10 0 0 0 0 DOUBLE REF same type passed 10 0 0 0 0 DOUBLE REF consistent passed 10 0 0 0 0 DOUBLE REF standard is equal passed 7 3 0 0 0 DOUBLE REF equal failed 5 0 0 0 5 DOUBLE REF standard equal failed 3 0 0 0 7 DOUBLE REF deep equal failed 3 0 0 0 7 DOUBLE REF copy passed 9 1 0 0 0 DOUBLE REF standard copy passed 9 1 0 0 0 DOUBLE REF clone passed 10 0 0 0 0 DOUBLE REF standard clone passed 10 0 0 0 0 DOUBLE REF standard twin passed 10 0 0 0 0 DOUBLE REF deep clone passed 10 0 0 0 0 DOUBLE REF deep copy passed 7 3 0 0 0 DOUBLE REF setup passed 10 0 0 0 0 DOUBLE REF io passed 10 0 0 0 0 DOUBLE REF tagged out assed 10 0 0 0 0 DOU
27. all features of all classes of the cluster to the scope add class adds all features of the class add features adds all features in the list and add feature adds just one feature to the scope These features do not only add features to the scope they also set the query is in test of the features to True This is later needed to create the context the pool of objects to perform the test feature calls 4 2 2 EXHAUSTIVENESS The attribute exhaustiveness in the class TW FEATURE determines the number of times the feature under test should be called Class INFORMATION HANDLER contains four features to set the exhaustiveness set globally set cluster set class and set feature Using these features the setting can be done at several levels for all features for all features of a cluster for all features of a class and per feature Furthermore the class TW FEATURE contains a query is descendants set which informs if the feature is tested in descendants The Information handler provides the feature set descendants to set this query to True 4 2 3 CONTEXT The attribute context of type ARRAYED LIST OBJECT contains the pool of objects Four features are at disposal to fill this structure with the needed tw objects to perform the selected test feature calls Here the class TW OBJECT described before is needed The explanation how the tw objects are added to the context will start at the lowest level the feature add to context T
28. can be found in actual generic parameters that has the type ARRAYED LIST TW 22 Test Wizard Master Thesis CLASS TW OBJECT As the name suggests this class represents objects namely instances of classes Every feature needs objects for its class and arguments Initially instances of the class TW OBJECT are not used therefore the Eiffel analyzer does not deal with this class Nevertheless this class is used to store required information even if the information is just retrieved from the other classes described before This is the reason why the class is already mentioned here The object representing the class is stored in the attribute class object of class TW FEATURE For the arguments the class TW TYPE has an attribute object to store the object representing the arguments The class contains the following information the associated class in the attribute associated class of type STRING the formal generic parameters as well as the actual generic parameters in formal generic parameters and actual generic parameters both of type ARRAYED LIST TW the creation procedure the modifiers and the descendants special attribute of this class is bounds which is of type ARRAYED LIST ANY It holds the user defined bounds which will be explained later Furthermore the class contains the queries is deferred and is expanded 4 1 2 CLASS TW ANALYZER The main feature of class TW ANALYZER is analyze all First the featur
29. class Object Instance of a class created during execution The retrieved system information contains the list of clusters deferred and effective classes their generic parameters and descendants deferred and effective features their arguments and result types as well as modifiers and creation procedures 3 2 INFORMATION HANDLER The information handler receives the system information and enables the user to choose different test criteria The class TW INFORMATION HANDLER implements the information handler The GUI development was not part of the project To highlight the changes between the project description and the actual development I used the following marks Y No changes e Additional implementation o Notimplemented 15 16 Test Wizard Master Thesis user set five test criteria e Scope of the test v v Clusters Classes List of features Features e Exhaustiveness Y Y Number of calls performed on each feature Y Globally Y Cluster Y Class Y Feature Testing the same feature in descendants Context Y 5 5 5 User defined bounds To define a bound the user has to write a feature that returns the bound a given type in a given state Furthermore he has to put the feature in a text file called bounds placed in the directory also called bounds of the Test Wizard delivery Then this file is put in the generated code and called during the initializ
30. class TW CONTEXT that consists of the creation procedure make The creation is the second part of the generated class TW CONTEXT The command generate all creation called by the feature generate context class see Figure 11 generates this part of the class This command relies on eight features The feature generate void creation generates the code of the feature create void object of the class TW CONTEXT The correct name of this feature should be create default initialization value object because expanded types cannot be Void The following table shows the default initialization values of the different types Type Default value INTEGER REAL DOUBLE 0 BOOLEAN False CHARACTER Null Reference types Void Table 6 Default initialization values Local entities are initialized with these values therefore local entities are declared for expanded types The query is expanded type is used to differentiate between expanded and reference types the containers are filled with an appropriate object initialized with the default value using Void for reference types The command generate all creation differentiates after the generation of the Void objects between effective and deferred types type ANY and on top of that it 40 Test Wizard Master Thesis distinguishes effective types between number BOOLEAN CHARACTER or reference types Number types include the type INTEGER INTEGER 5 INTEGER 16 INTEGER 64
31. infix lt passed 9 1 0 0 0 STRING infix gt passed 10 0 0 0 0 STRING infix gt passed 9 1 0 0 0 STRING three way comparison passed 7 3 0 0 0 STRING max passed 8 2 0 0 0 STRING min passed 9 1 0 0 0 STRING area passed 10 0 0 0 0 STRING mismatch information passed 10 0 0 0 0 90 1 2 3 4 5 6 7 8 9 10 11 12 13 Test Wizard Master Thesis 10 REFERENCES Karine Arnout Xavier Rousselot Bertrand Meyer Test Wizard Automatic test case generation based on Design by Contract draft report ETH June 2003 Retrieved July 2003 from http se inf ethz ch people arnout rousselot meyer test wizard pdf ric Bezault Gobo Eiffel Project Retrieved June 2003 from http www gobosoft com eiffel gobo index html Eric Bezault Gobo Eiffel Lint Retrieved June 2003 from http cvs sourceforge net cgibin viewcvs cgi gobo eiffel gobo src gelint Eric Bezault Gobo Eiffel Test Retrieved June 2003 from http www gobosoft com eiffel gobo getest index html Robert V Binder Testing Object Oriented Systems Models Patterns and Tools Addison Wesley 1999 Bertrand Meyer Applying Design by Contract Technical Report TR EI 12 CO Interactive Software Engineering 1986 Published in EEE Computer vol 25 no 10 October 1992 p 40 51 Bertrand Meyer Eiffel The Language Prentice Hall 1992 Bertrand Meyer
32. keep the index Furthermore I have used the class PLAIN FILE to generate the code that offers all features needed I only had to generate the code in order because it is not possible to write in the middle of a file At least if you do not want to overwrite already written code The advantage 15 that features generating the code can be read step by step to comprehend how the code is generated The disadvantage is that I have to loop over the context for every part of the class CONTEXT see Figure 12 Finally I would have saved a lot of time if it would be possible in EiffelStudio to run the program only without breaks caused by exceptions above all if they are rescued 6 2 THE SYSTEM INFORMATION Classes for which gelint encountered a problem are not processed by the Test Wizard because it is not possible to retrieve the system information correctly For example if gelint found a syntax error in a class the class should be reviewed first The Test Wizard writes the class names to the standard output so that the user has a list with all the excluded classes Furthermore the list contains additionally all the obsolete classes because they are excluded as well the features that use an object of an obsolete type as an argument or result type are logically also excluded The obsolete features are on the contrary tested 57 Test Wizard Master Thesis 6 3 CONTEXT Which objects are needed to test a cluster a cla
33. of the type i e the class name for the type he wants to define a bound and the name of the feature he has implemented It is also possible to replace the default creation procedure of an object in the context using the feature set creation procedure The object class name and the new creation procedure have to be provided Furthermore the level of randomness of targets and arguments can be set It is defined in the attribute randomness level and be changed by the feature set randomness level This feature sets the randomness level to the new level wanted At the beginning the randomness level is set to the Default randomness level which 15 1 constants used can be found in the feature clause Constants 28 Test Wizard Master Thesis 4 2 4 TOLERANCE For all different assertion checking levels an attribute of type BOOLEAN is implemented The attributes are found in the feature clause Status report and they are named as follows preconditions postconditions class invariants loop invariants loop variants and check instructions All of these attributes are associated with two features If the feature names start with set then the feature sets the appropriate attribute to True meaning the checking of this assertion is turned on otherwise the names start with set no These features set the appropriate attribute to False meaning that the checking of this assertion is turned off The default tolerance is repres
34. one or more external features Figure 3 The class TW CODE GENERATOR and its ancestors 17 Test Wizard Master Thesis classes used to implement the code generator are descendants of the classes TW CODE GENERATOR CONSTANTS and TW CODE GENERATOR SUPPORT These classes offer constants and support features for example features to generate common pieces of code like the inherit part The class TW CONTEXT GENERATOR SUPPORT contains support features for the class TW CONTEXT GENERATOR The class CONTEXT GENERATOR includes the code generator the context handler and the context generator I will call it context generator to simplify matters It generates the file containing the class TW CONTEXT that represents the pool of objects Once the context generator has established the order and means to instantiate the needed classes it actually generates the objects and calls modifiers randomly on those objects to build the context The generated class TW CONTEXT creates for each class a container holding objects of this class The container is filled in three steps Put one Void object in every container respectively the initialization value for container of objects of expanded types e terate and create some objects of each type e Call modifiers on the created objects The duplicates in the container are deleted afterwards to avoid that a container holds the same object twice or even more The default creation procedure is chos
35. peg 29 4 2 5 Testing x 29 4 3 Code E 30 4 3 1 SUDDOPLISSSE ai ode atu be 30 4 3 2 Class TW CONTEXT GENERATOR miter 31 4 3 3 Class TW TEST GENERATOR 22245 eut 44 4 3 4 Class TW CODE GENERA TOR a satt pk Pe 50 5 Test OF 1 2 c 53 5 1 The feature purge_duplicates 53 52 Bas types 53 9 3 Not test ble Classes nce 54 54 Clust r Co NIC I COE 55 57 6 1 Eiffel language and 12 2 57 6 2 The system information eoo sed 57 6 3 enn a 58 6 3 1 Creation 59 6 3 2 o 20 6 3 3 Arguments and generic paramieters 60 5 o cc 61 Test Wizard Master Thesis 7 enim em
36. put the objects in topological order The first command assigns an integer code to every object that is later needed to sort the objects 34 Test Wizard Master Thesis The class TW OBJECT offers two attributes name code and argument code as well as two commands set name code and add argument code to assign a code to every object The command goes first through all objects in the context and sets the name code for them using the feature create code Every time the feature is called it adds one to the integer stored in the attribute actual number using the feature counter and sets the name code to the new integer stored in actual number Finally every object is associated with another code Then all codes of the arguments of the creation procedure are put in the list argument code provided that the object represents a class that has a creation procedure respectively a creation procedure is needed to create an object of that type This attribute is in the class OBJECT The sorting algorithm used is quite simple It loops through the context and looks at the biggest argument code that is in the list argument code using the feature biggest argument code of class TW OBJECT The creation procedure has an object as argument that is initialized later if this argument code is bigger than the name code In this case the object is removed from the list and replaced in the list after the required object for the creation Table 3 and Table 4
37. show a short example to demonstrate the sorting algorithm Class name Creation procedure Name code Argument codes ARRAY make min index index INTEGER I 4 4 BOOLEAN 2 STRING make_filled c CHARACTER n INTEGER 3 6 4 INTEGER 4 DOUBLE 5 CHARACTER 6 REAL 7 Table 3 Sorting Create and set the name codes 2 Setthe argument codes 3 Sort see Table 4 At the first step the object of type ARRAY is put after INTEGER because the creation procedure has objects of type INTEGER as arguments Nothing happens at the second step because the class BOOLEAN does not have a creation procedure At the third step the object of type STRING 15 placed after CHARACTER The creation procedure of this class needs two objects one of type CHARACTER and the other of type INTEGER Since the object of type CHARACTER is created later than the INTEGER this object is placed after that of type CHARACTER and so on 35 Test Wizard Master Thesis ARRAY BOOLEAN BOOLEAN BOOLEAN BOOLEAN BOOLEAN STRING STRING INTEGER INTEGER STRING INTEGER INTEGER ARRAY ARRAY INTEGER ARRAY ARRAY DOUBLE DOUBLE DOUBLE DOUBLE DOUBLE CHARACTER CHARACTER STRING STRING REAL REAL REAL REAL REAL Table 4 Sorting steps The information handler already takes care during the creation of the context that all the objects that are needed for the test featur
38. that means the feature name the associated class and the cluster name The feature has to be exported to any class or has to be a creation 23 Test Wizard Master Thesis procedure to be processed In addition the feature should not contain any type of an obsolete class First of all the feature is set to deferred if it represents a deferred feature Then the result type of the feature is set using the feature feature result type Afterwards the tw feature is set as modifier provided that it is not a feature of ANY that the feature is exported to any class and of course that the feature does not have a result type The next step consists in adding the arguments to the feature To do that the feature feature argument is used The feature feature result type as well as the feature feature argument are using the feature actual parameters to determine the actual generic parameters of the type Finally the tw feature is added to the tw class If the feature is a creation procedure it is treated in a special way see Figure 5 In that case the feature can be exported in two different ways first as creation procedure and second as feature of the class The feature is added if it is exported to any class The feature considered as creation procedure has another export status If the creation procedure is also exported to any class then a new feature is created field to field identical to the old one set as creation procedure and finall
39. the object is removed from the extra context and not added to the context Then the feature fill extra context iterates again through the context as long as no more additional objects are found and put in the extra context The feature terminates if the extra context is empty after looping over all objects in the context 36 Test Wizard Master Thesis The first part of the class CONTEXT is now generated calling the command generate initialization The command generates the creation procedure make of class TW CONTEXT The creation procedure consists of nine parts e Initialization of pseudo number sequence using a default seed e Container creation e Creation of void objects one void object for every container e Creation of an object of type ANY e Creation of predefined default bounds for basic types e Object initialization creation and modification of the objects e Creation of user defined bounds e Creation of objects of ANY using descendants e Purge duplicates The first part is simple It just writes one line of code It creates and initializes an object of type RANDOM and attaches it to the attribute random To generate the code for the following parts the schema in the Figure 12 is used It loops through all the objects in the context and differentiates between objects of deferred and effective classes Furthermore it checks if the class is part of the scope if it is an object of a deferred class A class is part of
40. type as argument is always replaced by an effective type The object declared of the deferred type is needed because of its actual generic parameters To differentiate the objects the object declared of the deferred type is called from now on deferred object and the object declared of the descendant type is called effective object First of all the actual generic parameters of the 41 Test Wizard Master Thesis deferred object are stored in the attribute actual parameters The next step see Figure 15 is to replace a formal generic parameter as argument by either an actual generic parameter of the deferred object or a formal generic parameter of the effective object The command search tag object that relies on the command search parameter tag is called It sets the attribute ast class found item It compares first the attribute parameter tag of the argument and the actual generic parameters of the deferred object afterwards it compares it with the generic parameter of the effective object If there is a match the replacing argument is found In the case that the argument is not a generic parameter i e no replacing argument is found then the local entity temp object is just set to the object of the argument if a argument parameter tag void then search tag object a argument parameter tag a object actual generic parameters a actual formal generic parameters temp object last class found item object from class else a a
41. void target object Possible catcall is a new case that I will explain later The case exception raised and caught internally described in the project plan is not handled The generation of the context and the test feature calls are independent The class TW CONTEXT offers access features context xxx and void context xxx to the pool of objects xxx replaced by the class name including the parameters Therefore the test feature calls can already be generated even if the context is not generated yet The ancestor of all these classes is the class ZW CODE GENERATOR It calls the Eiffel compiler which builds the actual test executable The test executable contains the necessary code to create the context the pool of test objects and the test calls 3 4 TEST RESULT HANDLER The test result is either passed could not be tested no call was valid or failed The test result handler provides the result in two different formats for the user Excel compatible text files and XML files The project did not include any GUI development therefore a graphical representation displayed by the GUI Test result displayer is not provided The files using the Gobo Eiffel Test format 4 are not provided either The test result handler is included in the code generator therefore no new class is implemented 19 20 Test Wizard Master Thesis Test Wizard Master Thesis 4 IMPLEMENTATION OF THE TEST WIZARD This chapter describes the implementati
42. 0 0 CHARACTER REF equal failed 4 0 0 0 6 0 CHARACTER REF standard equal failed 7 0 0 0 3 0 CHARACTER REF deep equal failed 7 0 0 0 3 0 CHARACTER passed 9 1 0 0 0 0 CHARACTER REF standard copy passed 10 0 0 0 0 0 CHARACTER REF clone passed 10 0 0 0 0 0 CHARACTER standard clone passed 10 0 0 0 0 0 CHARACTER REF standard twin passed 10 0 0 0 0 0 CHARACTER deep clone passed 10 0 0 0 0 0 CHARACTER deep copy passed 10 0 0 0 0 0 CHARACTER REF setup passed 10 0 0 0 0 0 CHARACTER REF io passed 10 0 0 0 0 0 CHARACTER REF tagged out passed 10 0 0 0 0 0 CHARACTER REF print passed 10 0 0 0 0 0 CHARACTER REF Operating environment passed 10 0 0 0 0 0 CHARACTER REF default rescue passed 10 0 0 0 0 0 CHARACTER REF default create passed 10 0 0 0 0 0 CHARACTER REF do nothing passed 10 0 0 0 0 0 CHARACTER REF default passed 10 0 0 0 0 0 CHARACTER REF default pointer passed 10 0 0 0 0 0 CHARACTER REF Void passed 10 0 0 0 0 0 Table 9 Test results of testing the class CHARACTER REF 73 74 Test Wizard Master Thesis lt xml version 1 0 encoding ISO8859 1 gt test results feature lt name gt item lt name gt lt class gt CHARACTER_REF lt class gt lt result gt passed lt result gt lt no_exception gt 10 lt no_exception gt lt precondition_violation gt 0 lt precondition_violation gt lt assertion_violation gt 0 lt assertion_violation gt lt other_exception gt 0 lt other_exception g
43. 0 0 0 0 STRING string passed 10 0 0 0 0 STRING substring index passed 4 6 0 0 0 STRING fuzzy index passed 1 9 0 0 0 STRING capacity passed 10 0 0 0 0 STRING count passed 10 0 0 0 0 STRING occurrences passed 10 0 0 0 0 STRING index_set passed 10 0 0 0 0 STRING is equal passed 10 0 0 0 0 STRING string passed 9 1 0 0 0 STRING infix lt passed 9 1 0 0 0 STRING has passed 10 0 0 0 0 STRING substring passed 7 3 0 0 0 STRING extendible passed 10 0 0 0 0 STRING prunable passed 10 0 0 0 0 STRING valid index passed 10 0 0 0 0 STRING changeable comparison criterion passed 10 0 0 0 0 STRING is integer passed 10 0 0 0 o STRING is_real passed 10 0 0 0 0 STRING double passed 10 0 0 0 0 STRING boolean passed 10 0 0 0 0 STRING set assed 7 3 0 0 0 STRING copy passed 9 1 0 0 0 STRING subcopy no call was valid 0 10 0 0 0 STRING replace_substring passed 1 9 0 0 0 STRING replace substring all passed 3 7 0 0 0 STRING replace blank passed 10 0 0 0 0 STRING fill blank passed 10 0 0 0 0 STRING fill with passed 10 0 0 0 0 STRING character passed 10 0 0 0 0 STRING fill character passed 10 0 0 0 0 STRING head passed 4 6 0 0 0 STRING head passed 8 2 0 0 0 88 Test Wizard Master Thesis
44. 0 0 0 0 0 STRING compare objects no call was valid 0 10 0 0 0 STRING compare references no call was valid 0 10 0 0 0 STRING generator passed 10 0 0 0 0 STRING generating type passed 10 0 0 0 0 STRING conforms to passed 10 0 0 0 0 STRING same type passed 10 0 0 0 0 STRING consistent passed 10 0 0 0 0 STRING standard is equal passed 6 4 0 0 0 STRING equal failed 2 0 0 0 8 STRING standard equal failed 1 0 0 0 9 STRING equal failed 2 0 0 0 8 STRING standard passed 6 4 0 0 0 STRING clone passed 10 0 0 0 0 STRING standard clone passed 10 0 0 0 0 STRING standard twin passed 10 0 0 0 0 STRING deep clone passed 10 0 0 0 0 STRING deep copy failed 5 2 3 0 0 STRING setup passed 10 0 0 0 0 STRING passed 10 0 0 0 0 STRING tagged out passed 10 0 0 0 0 STRING print passed 10 0 0 0 0 STRING Operating environment passed 10 0 0 0 0 STRING default rescue passed 10 0 0 0 0 STRING default create passed 10 0 0 0 0 STRING do nothing passed 10 0 0 0 0 STRING default passed 10 0 0 0 0 STRING default pointer passed 10 0 0 0 0 STRING Void passed 10 0 0 0 0 STRING Growth percentage passed 10 0 0 0 0 STRING Minimal increase passed 10 0 0 0 0 STRING additional space passed 10 0 0 0 0 STRING resizable passed 10 0 0 0 0 STRING automatic grow passed 10 0 0 0 0 STRING full passed 10 0 0 0 0 STRING hashable passed 10 0 0 0 0 STRING
45. 6 item no call was valid 0 10 TUPLE integer item no call was valid 0 10 TUPLE integer 32 item no call was valid 0 10 TUPLE integer 64 item no call was valid 0 10 TUPLE pointer item no call was valid 0 10 TUPLE real item no call was valid 0 10 TUPLE hash code passed 10 0 TUPLE index passed 10 0 TUPLE valid type for index no call was valid 0 10 TUPLE count passed 10 0 TUPLE lower passed 10 0 TUPLE upper passed 10 0 TUPLE is empty passed 10 0 TUPLE put no call was valid 0 10 TUPLE put reference no call was valid 10 ojojo TUPLE put_boolean no call was valid 10 TUPLE put character no call was valid 10 TUPLE put_double no call was valid 0 10 TUPLE put_real no call was valid 0 10 TUPLE put pointer no call was valid 0 10 eo ojoilojoj ojoio oj oj oj oj oj oj o ojojo o ojoj o oj ojo o ojooloo 4 O O O oO 86 Test Wizard Master Thesis
46. BLE REF print passed 10 0 0 0 0 DOUBLE REF Operating environment passed 10 0 0 0 0 DOUBLE REF default rescue passed 10 0 0 0 0 DOUBLE REF default create passed 10 0 0 0 0 DOUBLE REF do nothing passed 10 0 0 0 0 DOUBLE REF default passed 10 0 0 0 0 TI Test Wizard Master Thesis DOUBLE REF default pointer passed 10 0 0 0 0 DOUBLE REF Void passed 10 0 0 0 0 DOUBLE REF infix lt passed 10 0 0 0 0 DOUBLE REF infix gt passed 8 2 0 0 0 DOUBLE REF infix gt passed 9 1 0 0 0 DOUBLE REF max passed 8 2 0 0 0 DOUBLE REF min passed 10 0 0 0 0 REAL REF item passed 10 0 0 0 0 REAL REF hash code passed 10 0 0 0 0 REAL REF sign passed 10 0 0 0 0 REAL REF one passed 10 0 0 0 0 REAL REF zero passed 10 0 0 0 0 REAL REF infix passed 8 2 0 0 0 REAL REF is equal passed 8 2 0 0 0 REAL REF three way comparison passed 9 1 0 0 0 REAL REF Set item passed 10 0 0 0 0 REAL REF divisible passed 8 2 0 0 0 REAL REF exponentiable passed 9 1 0 0 0 REAL REF is hashable passed 10 0 0 0 0 REAL REF truncated to integer passed 10 0 0 0 0 REAL REF truncated to integer 64 passed 10 0 0 0 0 REAL REF ceiling passed 10 0 0 0 0 REAL REF floor passed 10 0 0 0 0 REAL REF rounded passed 10 0 0 0 0 REAL REF abs passed 10 0 0 0 0 REAL
47. EF clone passed 10 0 0 0 0 BOOLEAN REF standard clone passed 10 0 0 0 0 BOOLEAN REF standard twin passed 10 0 0 0 0 BOOLEAN REF deep clone passed 10 0 0 0 0 BOOLEAN REF deep copy passed 5 5 0 0 0 BOOLEAN REF setup passed 10 0 0 0 0 BOOLEAN REF io passed 10 0 0 0 0 BOOLEAN REF tagged out passed 10 0 0 0 0 BOOLEAN REF print passed 10 0 0 0 0 BOOLEAN REF Operating environment passed 10 0 0 0 0 BOOLEAN REF default rescue passed 10 0 0 0 0 BOOLEAN REF default create passed 10 0 0 0 0 BOOLEAN REF do nothing passed 10 0 0 0 0 BOOLEAN REF default passed 10 0 0 0 0 BOOLEAN REF default pointer passed 10 0 0 0 0 BOOLEAN REF Void passed 10 0 0 0 0 INTEGER REF item passed 10 0 0 0 0 INTEGER REF hash code passed 10 0 0 0 0 INTEGER REF sign passed 10 0 0 0 0 INTEGER REF one passed 10 0 0 0 0 INTEGER REF zero passed 10 0 0 0 0 INTEGER REF ascii char passed 6 4 0 0 0 INTEGER REF Min value passed 10 0 0 0 0 INTEGER REF Max value passed 10 0 0 0 0 INTEGER REF infix passed 10 0 0 0 0 INTEGER REF is equal passed 10 0 0 0 0 INTEGER REF three way comparison passed 10 0 0 0 0 INTEGER REF set item passed 10 0 0 0 0 INTEGER REF divisible assed 8 2 0 0 0 INTEGER REF exponentiable passed 9 1 0 0 0 INTEGER REF is hashable passed 10 0 0 0 0 INTEGER REF is valid character code passed 10 0 0 0 0 INTEGER REF abs passed 10 0 0 0 0 INTEGER REF infix passed 6 4 0 0 0 INTEGER REF infix passed 8 2 0 0 0 INTEGER REF infix passed 5 5 0 0 0 INTEGER REF infix
48. ELL 21 MULTI ARRAY LIST 22 PART SORTED LIST _TWO_WAY_LIST 5 cursors CURSOR 2 ARRAYED 3 COMPACT 4 LINKED 5 HASH_TABLE 6 CIRCULAR _LIST_CURSOR _TREE_CURSOR _TREE_CURSOR CURSOR CURSOR 7 TWO WAY 8 MULTAR 9 LINKED TREE CURSOR LIST CURSOR _LIST_CURSOR 6 storage 1 RESIZABLE 2 UNBOUNDED 3 COUNTABLE 4 BOUNDED 5 FINITE 6 FIXED 7 INFINITE 8 BOX 7 access 1 INDEXABLE 2 ACTIVE 3 BAG 4 COLLECTION 5 CURSOR_STRUCTURE 6 CONTAINER 7 TABLE 8 set 1 SET 2 LINKED SET 3 LINEAR SUBSET 4 PART 5 TRAVERSABLE 6 BINARY SORTED SET _SUBSET _SEARCH_TREE_SET 7 COMPARABLE 8 ARRAYED_SET 9 SUBSET 10 TWO_WAY _SET _SORTED_SET 9 traversing LINEAR 2 HIERARCHICAL 3 TRAVERSABLE 4 BILINEAR 10 support DEBUG OUTPUT 2 MEMORY 3 MISMATCH 4 PRIMES 5 RANDOM 6 COUNTABLE STRUCTURE CORRECTOR SEQUENCE 7 FIBONACCI 8 FORMAT INTEGER 9 MEMORY 10 ASCII 11 CONST 12 FORMAT DOUBLE STREAM 3 C STRING 11 support classic INTERNAL 2 OPERATING 3 DOUBLE MATH 4 MANAGED POINTER 5 MISMATCH 6 BOOL STRING ENVIRONMENT INFORMATION 7 IDENTIFIED 8 PROFILING 9 EXECUTION 10 IDENTIFIED 11 SINGLE_MATH 12 CLASS NAME SETTING _ENVIRONMENT CONTROLLER TRANSLATIONS 13 Test Wizard Master Thesis 12 table 1 HASH TABLE 2 ARRAY2 13 tree 1 DYNAMIC TREE 2 LINKED TREE 3 TWO WAY TREE 4 BINARY 5 TREE 6 BINARY TREE SEARCH 7 FIXED TREE 8 ARRAYED TREE 14 cursor tree 1
49. ER 8 REF abs failed INTEGER 8 infix failed INTEGER 8 REF infix failed INTEGER 8 REF infix failed INTEGER 8 REF infix failed INTEGER 8 REF _ prefix failed INTEGER 8 REF prefix passed 1 INTEGER 8 infix failed GO GO CO CO Ni Co O1 O1 Ol 4 CO BE NIT NY ME CO PO PO CO PO BY NI 00 O O O 4 BLO BRIN NJ Ol Go Po AININ BPO O O1 BR O CO O1 HY O 4 Po O S ITO CO PO BR BY BY A IO O1 CO olol 84 Test Wizard Master Thesis
50. F CONTENTS MEN sei e 1 MES 4r besso 3 2 1 Project cles io eT 2 2 1 1 Scope of the MT 3 2 12 liunc R 7 2 1 3 amp 2 14 9 2 1 5 Plan WA oeecoocdut en DD WPRUN DIO DNE DUUM d d 11 3 Design of the Test Wizard e 13 3 1 Eiffel analyzer 15 3 2 Information handle 15 3 3 Code generator isser geen oir AE aE EEE NEAN E RE EER E E R EE iSS 17 i licher 19 4 Implementation of the Test Wizard eere eese eese eee 21 4 Eiffel Analyzer P 21 4 1 1 System information storage 2 21 41 2 Class TW uU AP GMT 23 42 Informati n handler senesnes ed mu Ep redu eMe 25 4 2 1 SCOPE CIMA TN 25 4 2 2 Exhaust VEn SS yi 25 4 2 3 onu 25 4 24 Toletan ga
51. Figure 7 returns True This is the case either if a feature of the class is under test and the class is a deferred class or if a feature is set to be tested in descendant classes as well 27 Test Wizard Master Thesis is in test a class TW CLASS BOOLEAN is Is feature of a class in test do from a class features start until Result or a class features after loop Result a class features item is in test and class is deferred or a class features item is descendants set a class features forth end end Figure 7 Feature is in test of class TW INFORMATION HANDLER The highest level represents the feature add cluster to context it adds all tw objects that a cluster needs to be tested This is simply done by going through all the tw classes and using the feature one level down It is possible for the user to define bounds These user defined bounds are directly added to the context by the feature add bound a name STRING a bound ANY To define a bound the user has to write a feature that returns a bound A bound is a given type in a given state The feature written has to be put in a text file called bounds placed in the directory called bounds as well Figure 8 shows an example how the file bounds could look like thousand INTEGER is do Result 1000 end hundred INTEGER is do Result 100 end Figure 8 An example file of user defined bounds The user only has to provide the name
52. REF infix passed 9 1 0 0 0 REAL REF infix passed 8 2 0 0 0 REAL REF infix passed 9 1 0 0 0 REAL REF infix passed 8 2 0 0 0 REAL REF infix passed 10 0 0 0 0 REAL REF prefix passed 10 0 0 0 0 REAL REF prefix passed 10 0 0 0 0 REAL REF out passed 10 0 0 0 0 REAL REF generator passed 10 0 0 0 0 REAL REF generating type passed 10 0 0 0 0 REAL REF conforms to passed 7 3 0 0 0 REAL REF same type passed 10 0 0 0 0 REAL REF consistent passed 10 0 0 0 0 REAL REF standard is equal passed 9 1 0 0 0 REAL REF equal failed 3 0 0 0 7 REAL REF standard equal failed 3 0 0 0 7 REAL REF deep equal failed 2 0 0 0 8 REAL REF copy passed 9 1 0 0 0 REAL REF standard copy passed 8 2 0 0 0 REAL REF clone passed 10 0 0 0 0 REAL REF standard clone passed 10 0 0 0 0 REAL REF standard twin passed 10 0 0 0 0 REAL REF deep clone passed 10 0 0 0 0 REAL REF deep copy passed 9 1 0 0 0 REAL REF setup passed 10 0 0 0 0 REAL REF io assed 10 0 0 0 0 REAL REF tagged out passed 10 0 0 0 0 REAL REF print passed 10 0 0 0 0 REAL REF Operating environment passed 10 0 0 0 0 REAL REF default rescue passed 10 0 0 0 0 REAL REF default create passed 10 0 0 0 0 REAL REF do nothing passed 10 0 0 0 0 REAL REF default passed 10 0 0 0 0 REAL REF default pointer passed 10 0 0 0 0 REAL REF Void passed 10 0 0 0 0 REAL REF infix lt passed 5 5 0 0 0 REAL REF infix gt passed 10 0 0 0 0 78 Test Wizard Master Thesis
53. T TYPE CONTEXT to base type actual not void Result Void definition Result same named type base type universe actual parameters type i root context root context universe named type named Result is named type This is already fixed in the CVS version of Gobo The version 5 4 or later of EiffelStudio is required to use the Test Wizard The assertion checking is otherwise not enabled and disabled correctly 67 Test Wizard Master Thesis 8 2 COMMANDS The Test Wizard is delivered as a demo application If you compile and run it with the file librarybase ace as argument the demo application tests the class BOOLEAN REF of the EiffelBase library The Test Wizard is not completely user friendly at the moment because there is no graphical user interface yet You have to set the test parameters in feature analyze universe of the root class TW TEST WIZARD see Figure 28 analyze universe a universe ET UNIVERSE is Analyze universe local analyzer TW ANALYZER info handler TW INFORMATION HANDLER code generator TW CODE GENERATOH a cluster TW CLUSTEH a class TW CLASS do a universe error handler set ise a universe set use attribute keyword True a universe set use convert keyword True a universe set use recast keyword False a universe set use reference keyword True a universe set use assign keyword True a universe activate processors a universe parse all a universe compi
54. TWO WAY 2 RECURSIVE 3 CURSOR TREE 4 LINKED 5 COMPACT CURSOR TREE CURSOR TREE CURSOR TREE CURSOR TREE 15 dispenser 1 LINKED STACK 2 STACK 3 LINKED PRIORITY QUEUE 4 PRIORITY QUEUE 5 HEAP PRIORITY QUEUE 6 BOUNDED QUEU E 7 QUEUE 8 DISPENSER 9 ARRAYED QUEUE 10 BOUNDED STACK 11 ARRAYED STACK 12 LINKED QUEUE 16 iteration 1 CURSOR TREE 2 LINEAR ITERATOR 3 ITERATOR 4 TWO WAY _ITERATOR _CHAIN_ITERATOR 17 strategies 1 SUBSET 2 SUBSET 3 SUBSET 4 SUBSET _STRATEGY _STRATEGY_HASHABLE STRATEGY TREE STRATEGY GENERIC 15 sort l COMPARABLE 2 SORTED STRUCT STRUCT 76 Table 10 Indexes of clusters and classes 9 2 TEST RESULTS Test Wizard Master Thesis 9 2 1 THE REFERENCE TYPES OF THE BASE TYPES no precondition assertion possible class feature result exception violation violation exception catcall DOUBLE REF item passed 10 0 0 0 0 DOUBLE REF hash code passed 10 0 0 0 0 DOUBLE REF sign passed 10 0 0 0 0 DOUBLE REF one passed 10 0 0 0 0 DOUBLE REF zero passed 10 0 0 0 0 DOUBLE_REF infix lt passed 7 3 0 0 0 DOUBLE_REF is equal passed 9 1 0 0 0 DOUBLE REF three way comparison passed 8 2 0 0 0 DOUBLE REF Set item passed 10 0 0 0 0 DOUBLE
55. Test Wizard Automatic test generation based on Design by Contract Master Thesis Nicole Greber Supervised by Prof Dr Bertrand Meyer and Karine Arnout ETH Z rich July 21 2003 January 21 2004 Test Wizard Master Thesis ACKNOWLEDGEMENTS First of all I want to thank Karine Arnout for her supervision her scientific support and her very helpful comments and reviews on this master thesis and Prof Dr Bertrand Meyer who made this master thesis possible Further I want to thank Xavier Rousselot for his ideas and support concerning the design and implementation of the tool I thank also Eric Bezault for his support with the tool Gobo Eiffel Lint Finally I am deeply grateful to my parents for their support in every respect ii iii Test Wizard Master Thesis Test Wizard Master Thesis ABSTRACT The project is based on the benefits of Design by Contract on testing If contracts preconditions postconditions class invariants are systematically associated with classes they provide an invaluable source of information for producing systematic tests directly based on the software s expressly intended semantics The goal of this project was to develop a tool called Test Wizard which generates test cases automatically from contract equipped classes The Test Wizard should serve as a workbench to try out different testing strategies iv Test Wizard Master Thesis Test Wizard Master Thesis TABLE O
56. The Test Wizard is not able to test base types respectively expanded types Features that use an anchored type as argument can cause runtime panic if an expanded type is tested An example is the feature copy of class ANY If the feature is called on an object of type INTEGER it causes a segmentation violation The Figure 22 shows the Eiffel code on the left side and the generated C code on the right side local EIF INTEGER 32 EIF INTEGER 32 30L int INTEGER RTLN RTUD 127 EIF INTEGER 32 tp1 loct 30 int 30 FUNCTION CAST void EIF REFERENCE EIF INTEGER 32 int copy int RTVF 127 14 copy tp1 tp1 loc1 int copy int end Figure 22 Feature copy of class ANY on expanded types 53 Test Wizard Master Thesis Xavier Rousselot commented on the generated code As you can see an INTEGER REF is created on the fly to be able to call copy which is normal RTLN allocates the memory for the INTEGER REF it is assigned the value of 30 and then is called Note that the second parameter of is an INTEGER not an INTEGER REF This is completely wrong because expects to be called with a real Eiffel object not an integer Therefore is considered to be a pointer to an Eiffel object whereas it is an And of course type fails it cannot retrieve the type information on an object that does not exist The funny conse
57. ake filled make from array feature Purge purge duplicates is Delete duplicates in Current local failure BOOLEAN a index INTEGER do compare objects from start until after loop if failure then failure False go i th a index forth else a index index if occurrences item 1 then remove else forth end end end rescue failure True print problems in purge duplicates N print Tclass print class name print N Tfeature print recipient_name print N Ttag print tag_name print N retry end end Figure 10 Class TW_CONTAINER 32 Test Wizard Master Thesis GENERATION OF CLASS TW CONTEXT The second feature namely generate context class creates the file tw context e containing the class TW CONTEXT This class creates the pool of objects needed to perform the test feature calls and looks therefore different depending on which features are tested TW CONTEXT is the most complex generated class and is divided in seven parts Initialization e Creation Modification e Special bounds e User defined code e Container e Access feature calls in the class CONTEXT are wrapped in a rescue block but the exceptions are not handled so that the generated program does not fail generating the context The creation and the modification are separated Failing to create the object simply because a precondition was violated in the modifier is avoided by this s
58. ard Master Thesis Finally the attribute ast object found item is set when calling the feature search object Both the feature and the attribute are implemented in class TW CODE GENERATOR SUPPORT feature searches the object in the context and sets last object found item to this object In case the found object is of type TUPLE the feature is not finished yet The class TUPLE is the only class which has a variable number of generic parameters Therefore if the actual parameters include the type TUPLE a new object is generated with the formal and actual generic parameters set to the actual generic parameters of the parameter TUPLE hold in the attribute actual parameters The part containing the creation of the user defined bounds is generated by the command generate all bounds that relies on generate bound In the class TW OBJECT there 15 an attribute bounds the list of user defined bounds This attribute contains all feature names that the user has implemented The code generated by this feature just assigns a variable to the feature and adds this variable to the appropriate container see Figure 17 create integer bounds is Create user defined bounds local integer variable1 INTEGER failure INTEGER do if failure 10 then integer variable thousand integer objects force integer variable else print Couldn t create object of class INTEGER N end rescue failure failure 1 retry end Figur
59. are failures No business wants an outrage to make the front page of the morning newspaper Because the cost of testing and verification can exceed the cost of design and programming the methodologies techniques and tools used for testing are key to efficient development of high quality software According to the NIST report on software testing the worldwide market for software testing tools was 931 million in 1999 and is projected to grow over 2 6 billion by 2004 Reuse is the key attribute in component based software engineering Robert Binder asserts that Components offered for reuse should be highly reliable 1 therefore testing gains even more importance Reducing the cost of software development and as a part of it reducing the cost of testing and improving software quality are important objectives of software engineering The project is based on the benefits of Design by Contract in testing 6 9 11 The information given by contracts like preconditions postconditions and class invariants is highly relevant for testing The contracts make it possible to produce systematic tests by using only the semantics of software If contracts preconditions postconditions class invariants are systematically associated with classes they provide an invaluable source of information for producing systematic tests directly based on the software s expressly intended semantics The goal of this project was to develop a tool called Tes
60. ation of the pool of objects Objects on which features are called Creation procedure used for instantiation Level of randomness of targets and parameters Fixes only the number of variants to be created not the final number in the container see Chapter 4 3 2 e Tolerance Allow rescued exceptions Can be set but it is not handled Partial checking of assertions Y Check instructions Y Precondition Y Postcondition Y Loop invariants and variants only both or none v Class invariants Test Wizard Master Thesis e Testing order Y Perform tests one feature at a time Y Perform tests on one class at a time v Test as many features as possible 3 3 CODE GENERATOR The code generator of Figure 2 covers the code generator the context handler the context generator the call simulator and the test result handler of Figure 1 Six classes implement the code generator TW CODE GENERATOR TW CODE GENERATOR CONSTANTS TW CODE GENERATOR SUPPORT TW CONTEXT GENERATOR SUPPORT TW CONTEXT GENERATOR and TW TEST GENERATOR Figure 3 shows their dependencies The first architecture proposed to generate the test executable at first and afterwards to establish the order and means to instantiate the needed classes In my opinion it is easier to generate the code right from the start than to generate the code and change it afterwards This is the reason why the architecture has changed inherits interfaced class has
61. atures of class ANY The features of class ANY are excluded because features like for example copy and print are not really interesting The 59 Test Wizard Master Thesis disadvantage of this approach is that there are features used as modifiers that do not change the state of the object at all The next problem was how do I access the different modifiers First I thought about putting all the modifiers in a list using agents so that I have the structure ARRAYED LIST ROUTINE ANY TUPLE But it turned out to be not so easy because of the infix features and arguments Finally I decided to use inspect instead Figure 25 shows a model of the feature that modifies the state of an object of the class XXX The modification features try ten times to modify an object In the case that they do not succeed in modifying an object a notice is written to the standard output the console The features choose the modifiers randomly using a random number and inspect The modifiers never called on void objects therefore it uses only the non void context xxx attribute modify xxx objectis local failure INTEGER do if failure 10 then random forth inspect 1 random item N m when 1 then non void context xxx feature 1 when n then non void context xxx feature n when m then non void context xxx feature m end else print Couldn t modify object of class XXX N end rescue failure failure 1 retry end Figure 25 O
62. bject modification 6 3 3 ARGUMENTS AND GENERIC PARAMETERS I had the most difficulties with the arguments and the parameter handling of generic arguments The majority of the compile errors were caused by wrong arguments and most of the time just wrong generic parameters of the argument 60 Test Wizard Master Thesis A special case of the parameter handling is if only one actual generic parameter replaces a formal generic parameter but the class has more than one formal generic parameter The parameters have to be merged together i e only the formal generic parameter for which an actual generic parameter exists must be replaced First of all the generic parameters of the generic parameters had to be handled as well Secondly the class TUPLE had to be treated in a special way because it has a variable number of generic parameters Another issue was the parameter handling if an effective class replaces a deferred class I also had to handle the fact that descendants of deferred classes can be deferred as well Furthermore classes can be inherited with an actual generic parameter For example the class SEQ STRING inherits the class SEQUENCE CHARACTER Therefore an object of type SEQ STRING can only be assigned to an object corresponding to the class SEQUENCE G if the generic parameter is CHARACTER Another problem was the class FUNCTION The class inherits from the deferred class ROUTINE but has one generic parameter more t
63. ble in the database Store the test results into a database Display the results graphically with diagrams Test Wizard Master Thesis 3 1 EIFFEL ANALYZER The Eiffel analyzer parses a contract equipped library provided as input to retrieve the system information The Eiffel analyzer is implemented by the class ANALYZER It uses the parsing tool Gobo Eiffel Lint called gelint 2 3 to have access to the required system information Gelint uses the tools Gobo Eiffel Yacc and Gobo Eiffel Lex It is able to analyze Eiffel source code and report validity errors and warnings The class TW ANALYZER uses as input the universe UNIVERSE generated by the tool The generation of the universe takes place in the root class called TEST WIZARD The required system information is stored an ARRAYED LIST of TW CLUSTER Five classes are needed to store the system information namely TW CLUSTER TW CLASS TW FEATURE TW TYPE and TW OBJECT The Eiffel language 7 is taken as a model to store the system information Cluster a group of related classes e Class the description of a type of run time data structures characterized by common features and attributes e Feature an operation available on instances of a class A feature can be either an attribute or a routine which is further classified into functions and procedures Functions return a result and procedures do not Type Every type is based on a class the type s base
64. cannot be tested and the classes POINTER POINTER REF and TYPED POINTER are not tested see chapter 5 3 Furthermore two classes produces all the problems that the Test Wizard cannot handle These problems are Infinite loops Anchored types that refer to a feature and not to an argument or the current type e Features calling C routines For the class STRING the Test Wizard encountered two difficulties First of all calling a feature that just calls a C routine and secondly an infinite loop in a feature The first problem is caused by the feature center justify and the second problem by the feature character justify This feature has an infinite loop if the pivot used for justifying the string is not in the string itself The feature index of returns in this case 0 and remains 0 No matter how many empty character are prepended see Figure 24 55 Test Wizard Master Thesis character justify pivot CHARACTER position INTEGER is Justify a string based on a pivot and the position it needs to be in the final string This will grow the string if necessary to get the pivot in the correct place require valid position position capacity positive position position gt 1 pivot not space pivot not empty not js empty do if index of pivot 1 position then from precede until index of pivot 1 position loop precede end elseif end end Figure 24 Code extra
65. ct of feature character justify of class STRING The class can be tested anyway but the user must comment the features that cause problems out In paragraph you can find the test results for class STRING without these two features The problem that anchored types referring to another feature are not catcall checked appears in the class ARRAY G The feature is inserted causes runtime panic The appendix shows an extract of the test results Most classes that should be excluded to test this cluster were mentioned before Still a few others needed to be excluded The generated project is adapted for the class ARRAY G and the class STRING 56 Test Wizard Master Thesis 6 DISCUSSION During my project I have gained a lot of experience in designing and implementing an Eiffel system and in using the tool gelint To share my experience I will discuss in this chapter the difficulties I have encountered during the design and implementation of the Test Wizard 6 1 EIFFEL LANGUAGE AND EIFFELSTUDIO I will start with the problems I had concerning the Eiffel language and EiffelStudio The container classes only support internal cursors but no external iterator In case of several traversals of a list in the same time the consistency must be ensured I used the class ARRAYED LIST G for lists and noticed that an instance of an ARRAYED LIST G had only an internal cursor Therefore I had to adapt all my features that handle lists so that they
66. d a command The generated creation procedure just sets the feature name and the class name calling the feature set feature name and set class name inherited from the deferred class TW TEST The generated command 15 the real feature test call This feature is called test feature and is already defined in the class TW TEST as a deferred feature Figure 19 shows the generated class for the call to feature max of class CHARACTER REF indexing description Test Wizard Automatic generated class date Date 2004 01 20 21 33 56 class TW TEST MAX inherit TW TEST create make feature Initialization make is Set feature name associated class name and exhaustiveness do set feature name max set class name CHARACTER REF set exhaustiveness 10 end feature Test feature test feature a context TW CONTEXT is Call feature feature name in context context Set exceptions local a var CHARACTER REF a current var CHARACTER REF other CHARACTER REF do a current var a context non void context character ref other a context context character ref if a current var void or other void or other conforms to a current var then var a current var max other set no exception else set possible catcall end end end Figure 19 Generated class TW TESTI MAX for feature test calls 48 Test Wizard Master Thesis The feature generate feature class first checks if the associate
67. d class of the feature is deferred or if the feature is set to be tested in descendants If one of these cases applies then the same feature of the descendant classes is also added to the scope The feature search feature looks for a feature with the same name in the descendant class to be added to the scope Therefore renaming is not considered In the case of an effective class the feature generate feature class assigns the feature name to the local entity a string to retrieve it later for the file and class name The feature class name replaces all tokens with a string for example lt lt is replaced by the string shift left so that the name only consists of characters Furthermore it appends at the beginning TEST and the name code of the associated object This guarantees that every file respectively class name is unique Then the code for the creation procedure is generated Finally the code for the feature fest feature is generated The code shown in Figure 20 differentiates between two cases either the feature is catcall checked when calling it or not It starts with assigning the attribute arguments to the feature arguments The feature set arguments the attribute be found in the class TW CODE GENERATOR SUPPORT A catcall is possible if the feature uses an anchored type as arguments The anchor can be e attribute or function of the enclosing class an argument of the enclosing feature e Current i e
68. d copy passed 9 1 0 0 0 INTEGER 64 REF clone passed 10 0 0 0 0 INTEGER 64 REF standard clone passed 10 0 0 0 0 INTEGER 64 REF standard twin passed 10 0 0 0 0 INTEGER 64 REF deep clone passed 10 0 0 0 0 82 Test Wizard Master Thesis INTEGER 64 REF deep passed 7 INTEGER 64 REF setup passed 10 INTEGER 64 REF passed 10 INTEGER 64 REF tagged out passed 10 INTEGER 64 REF print passed 10 INTEGER 64 REF Operating environment passed 10 INTEGER 64 REF default rescue passed 10 INTEGER 64 REF default create passed 10 INTEGER 64 REF do nothing passed 10 INTEGER 64 REF default passed 10 INTEGER 64 REF default pointer passed 10 INTEGER 64 REF Void passed 10 INTEGER 64 REF infix lt passed 10 INTEGER 64 REF infix gt passed INTEGER 64 REF infix gt passed INTEGER 64 REF max passed INTEGER 64 REF min passed INTEGER 16 REF item passed 1 INTEGER 16 REF hash code failed INTEGER 16 REF sign failed INTEGER 16 REF one failed INTEGER 16 REF zero failed INTEGER 16 REF ascii char failed INTEGER 16 REF Min value passed 1 INTEGER 16 REF Max value passed 1 INTEGER 16 REF infix passed INTEGER 16 REF is equal failed INTEGER 16 REF three way comparison failed INTEGER 16 REF set item failed
69. dant classes like ARRAY ANY are added The argument objects are also added if the feature is a creation procedure or a modifier but only if it is not a pointer because the pointers are too difficult to create automatically Therefore all creation procedures and modifiers needing a pointer are excluded from the context creation add feature to context a feature TW FEATURE is Add all objects that feature needs to context if not already included add arguments only if a feature is in test a feature is creation procedure or a feature is modifier is true require feature not void a feature void local arguments actual parameters ARRAYED LIST TW TYPE a class TW CLASS do a feature set objects a feature add class object add to context a feature class object arguments a feature arguments ifa feature is in testthen from arguments start until arguments after loop actual parameters arguments item actual generic parameters if not actual parameters is empty then from actual parameters start until actual parameters after loop actual parameters item set object add class to context actual parameters item associated class 26 Test Wizard Master Thesis actual parameters forth end end arguments item set object add to context arguments item object if not arguments item associated class is deferred then add class to context arguments item associated class else a class arguments i
70. e 17 Generated feature create integer bounds The next part the user defined bounds is generated by copying the file bounds that is placed in the directory bounds reading and writing one line after the other The remaining parts of the class TW CONTEXT are the container and the object access The code for the container access is generated by calling the feature generate all container that relies on features generate container and generate deferred container The one for the object access 15 generated by calling the feature generate all access relying on four features because there are two access possibilities The first object access returns an object of the container whereas the second access only returns non void objects from the container For every container both access possibilities are provided For example the modification of the created objects uses the 43 Test Wizard Master Thesis second access so that the modifiers are not called on void objects The non void access is generated by calling the features generate non void access and generate non void deferred access the other access is generated by calling the features generate access and generate deferred access The features that these features generates are used to access the pool of objects for the feature test calls The features are called context xxx or non void context xxx see Table 5 4 3 3 CLASS TW TEST GENERATOR The main features of class
71. e calls are in the context Anyway a few objects are needed and nevertheless not added yet The next command fill extra scope context calling the feature fill extra context repairs this The simple reason that these objects are not added on the level of the information handler is that the information handler does not care about the generic parameters at least not in a strict way This is only done in the code generator The feature fill extra scope context loops through the features in the scope and adds arguments of the feature to the extra context if the argument is a class with an actual generic parameter Afterwards the objects in the extra context are added to the context provided that the objects are not already in the context Then the feature fill extra context is called This feature relies on four features fill extra context add deferred extra context and add extra context and merge parameters These features are not explained in detail because they are based on the features that handle the creation and modification of the objects The pool of additional objects are stored in the attribute extra context in each iteration of the feature fill extra context and afterwards added to the context Before an object is added to the context it is checked if the object is already included in the list added objects to guarantee that no object is handled twice The list added objects contains all handled objects If the object is already handled then
72. e code generator generates Eiffel code corresponding to the defined test scenario and calls the Eiffel compiler which builds the actual test executable The context handler establishes the order and means to instantiate the needed classes The context generator actually generates the objects and calls modifiers randomly on those objects to build the context The call simulator wraps all calls in a rescue block to handle five possible cases no exception exception raised and caught precondition violation other assertion violation and other exception The test result handler provides the result in three different formats for the user Like mentioned before the project will not include any GUI development therefore only two formats are provided XML files and files using Gobo Eiffel test format 4 The database query handler gives the user the ability to query the database It is only partly implemented it should be possible at least to retrieve the test results more advanced functionalities beyond the scope of the project Storing into the database The demo shows the functionality and scope of the Test Wizard Test Wizard Master Thesis Documentation Developer manual The developer manual documents the software architecture and its limitations describes the difficulties encountered during the implementation and explains how to extend the current version User manual The user manual describes the usage of the T
73. e creates an empty structure and then it calls the features analyze clusters analyze classes analyze features and analyze descendants in order There are two access possibilities used by the Eiffel analyzer to access the structure UNIVERSE that gelint produces The first one is the attribute classes of type DS HASH TABLE ET CLASS ET CLASS NAME and the second is the attribute clusters of the type ET CLUSTERS The feature analyze clusters goes through all classes in the universe and gathers for each class the cluster namely the cluster s and the parent cluster s names The tw cluster is searched in the whole structure and created if it does not already exist Afterwards the feature checks if the class is deferred or expanded sets the tw class appropriately and adds the tw class to the tw cluster Finally the tw cluster is added to the structure but only if it was newly created Not every class is added to the structure there are some exceptions First of all if an error occurred while parsing the class in gelint secondly if the class is obsolete and thirdly if the class has only creation clauses that are not accessible from any class except for expanded classes The next step is the feature analyze classes It just adds the generic parameters of the class using the feature class generic parameter The feature analyze features goes through all features of the classes which are in the structure It gathers each tw feature
74. e directory also called bounds The demo application adds two user defined bounds namely the character m and n The file bounds contains therefore two features one returning the character m and the other the character see Figure 33 In the Figure 28 you see how these bounds are added using the feature add bound The demo application sets further the randomness level to 10 using the feature set randomness level and the context to the class CHARACTER REF 70 Test Wizard Master Thesis character m CHARACTER HEF is Create character m do create Result Result set item end character n CHARACTER is Create character n do create Result Result set item end Figure 33 The file bounds of the demo application The Figure 34 shows the features to enable or disable a certain level of assertion checking By default full assertion checking is enabled The demo application does not disable assertion checking therefore all assertions are checked at run time in the demo application As mentioned before to enable respectively disable the assertion checking correctly the version of EiffelStudio must be 5 4 The assertion checking is defined in the ace file of the generated project Loop invariants and loop variants are either both checked or both not checked Finally it is possible to allow or forbid rescue clauses in the test features You can define if the feature call is c
75. e queries shown in the Figure 30 cluster implementation a index INTEGER TW CLUSTER class implementation a index INTEGER a cluster TW CLUSTER TW CLASS feature implementation a index INTEGER a class TW CLASS TW FEATURE Figure 30 Features to retrieve clusters classes and features of the library under test The indexes needed by these features can be looked up in Table 10 This table shows the cluster and classes of the EiffelBase library The classes WIDE CHARACTER WIDE CHARACTER REF CONSOLE BIT REF and RECURSIVE CURSOR TREE cannot be tested by the Test Wizard because the wizard does not test e Obsolete classes e Classes that only have creation procedures that are not exported to ANY e Classes for which gelint encountered a parse error The demo application uses the feature add class to add the class CHARACTER REF to the scope The class is retrieved using the feature cluster implementation with index 1 Then the feature class implementation is called with the index 26 and the cluster just retrieved by the first feature see Figure 30 You can set the exhaustiveness with the features listed in Figure 31 Furthermore you can specify whether a feature should be retested in descendant classes The exhaustiveness is to retest no feature in descendants and 10 feature calls by default The demo application does not set the exhaustiveness therefore the default exhaustiveness 15 used 69 Test Wizard Master Thesi
76. ects of type POINTER It is not possible to generate objects of type POINTER 54 Test Wizard Master Thesis automatically because the probability that the program crashes is too high The context generation avoids as much as possible to use POINTER types Creation features as well as modifiers that use a POINTER type as argument are excluded if possible The deferred class FILE as well as the descendants of that class cannot be tested either As a consequence all the classes depending on that class cannot be tested either The user has to provide objects of that type like for the class POINTER Whereas the features of class POINTER are already excluded from the file generating the context these classes respectively their features are still in the file Therefore it is not possible to test classes concerned with the file handling at all with the current version of the Test Wizard Furthermore the class EXCEPTIONS cannot be tested because of the feature call die The program terminates when calling this feature 5 4 CLUSTER KERNEL CLASSIC This paragraph describes how to test the cluster kernel classic of the EiffelBase library to give an impression how the tool works You can lookup in Table 10 that the cluster kernel classic contains 43 classes First of all the base types cannot be tested These include the classes BOOLEAN CHARACTER INTEGER REAL and DOUBLE Moreover the classes EXCEPTIONS FILE PLAIN TEXT FILE and RAW FILE
77. en so that no current or pointer type is an argument The object creation of base types and their associated reference type is treated differently Predefined values are used to fill the containers These predefined bounds are listed in Table 2 An exception is the class INTEGER which is filled with the values 1000 100 10 5 4 3 22 1 0 1 2 3 4 5 10 100 1000 and a few random generated numbers between 1000 and 1000 The reason why this class is treated specially is memory allocation It prevents to allocate big chunks of memory see Chapter 6 4 Class Predefined default bounds INTEGER 1 0 1 2 value max value REAL DOUBLE 1 0 1 2 3 14159265358979323846 2 7182818284590452354 BOOLEAN True False CHARACTER a d Table 2 Predefined default bounds for base types The class TW TEST GENERATOR includes the call simulator and the test result handler I will call it test generator because it generates the code for the test feature calls A class inheriting from the deferred class TW TEST is generated for every test feature It 18 Test Wizard Master Thesis is also the test generator that generates this class Exception handling is done by the class TW TEST feature calls are wrapped into a rescue block to handle five possible cases no exception precondition violation other assertion violation other exception possible catcall and
78. ented by a constant boolean that is True the assertions are set by default to the default tolerance therefore full assertion checking is turned on at the beginning 4 2 5 TESTING ORDER The last test criteria 15 the testing order There exist three testing orders that are represented by a constant integer All testing order is 1 performing tests on as many features as possible e Feature testing order is 2 performing tests on one feature at a time e Class testing order is 3 performing tests on one class at a time The default testing order which is also represented by a constant is set to all testing order The attribute testing order which contains the actual testing order is set at the beginning to this default value The feature set testing order allows to change the attribute zesting order Only valid testing orders are allowed 29 Test Wizard Master Thesis 4 3 CODE GENERATOR The code generator is implemented by six classes The dependencies are shown in Figure 3 4 3 1 SUPPORT CLASSES The class TW CODE GENERATOR SUPPORT inherits from the class EXECUTION ENVIRONMENT to make it possible to change the actual working directory and to compile and run the generated code other classes implementing the code generator except the support classes are descendants of the class TW CODE GENERATOR and have access to all features offered by this class The attribute current working directory and the feature change
79. eparation Furthermore if one modifier fails to modify an object another modifier is called The selection of the modifiers as well as the arguments is random The feature generate context class Figure 11 starts with assigning the files These are the file containing the user defined code for the bounds and of course the file tw context e the attributes that handle the files can be found under the feature clause File access The names of the files are also defined here as constants The initialization of the structure added objects takes place right after assigning the files This structure is an ARRAYED LIST TUPLE STRING ARRAYED LIST TYPE The list is used to hold already added objects to the context to avoid adding an object twice The objects are represented by their names and generic parameters The generic parameters are available as an ARRAYED LIST TW Two queries is included and check included and a command add object handle this structure The queries are to check if the object is already included they just have a different signature In fact the feature check included relies on the feature is included The command as the name suggests just adds objects to the structure 33 Test Wizard Master Thesis generate context class is Generate context file local a directory STRING do create bound file make current working directory slash bound directory name slash bound file name c
80. eplacing the formal generic parameter In this case the formal generic parameter is replaced by the actual generic parameter The three other features under the same feature clause do exactly the same except that they produce a string representing the Eiffel code for generic parameters An example of the resulting string of these features are the generic parameters of the first column in Table 5 The following table shows two code extracts from the feature type list string They show the handling of the formal generic parameters of the formal generic parameters The class TUPLE 18 separately handled because it is the only class that has a variable number of actual generic parameters 38 Test Wizard Master Thesis a parameter ARRAYED LIST TW if a parameter item name is equal TUPLE then if not a parameter item actual generic parameters is empty then Result append type list string a parameter item actual generic parameters end end if not a parameter item associated class formal generic parameters is empty then if not a parameter item actual generic parameters is empty then Result append type list string a parameter item actual generic parameters else Result append type list string a parameter item associated class formal generic parameters end end Figure 13 Code extract from feature type list string of class TW CODE GENERATOR SUPPORT The container creation of objects of deferred classes 15 differe
81. est Wizard Intermediary report The intermediary report consists of the intermediary developer manual Thesis report The thesis report consists of the final developer manual the final user manual and a theoretical part discussing the profiling like the performance of the tool comparing automatic and manual testing etc 2 1 3 BACKGROUND MATERIAL READING LIST e Chapters in OOSC2 9 in particular o Chapter 1 Software quality Chapter 10 Genericity Chapter 11 Design by Contract building reliable software Chapter 12 When the contract is broken exception handling Chapter 14 Introduction to inheritance Chapter 15 Multiple inheritance Chapter 26 A sense of style o Chapter 28 The software construction process Testing Object Oriented Systems Models Patterns and Tools 5 Test Wizard Master Thesis 2 1 4 PROJECT MANAGEMENT OBJECTIVES AND PRIORITIES Objective Priority Software architecture Information handler Test scenario handler only implemented if time permits Code generator Context handler Context generator Call simulator Test result handler Database query handler only partly implemented Database storing Demo application Developer manual User manual Intermediary report Thesis report RINE MOR e Re CRITERIA FOR SUCCESS The criteria for success is the quality of the so
82. fail since the precondition array exists Void of the feature would not be fulfilled 6 3 2 MODIFIERS Which features are suitable as modifiers The features I used as modifiers changed during my project not only once The first approach was to use all features that have the current type as return type as well as all features under the feature clause Element change With the benefit of hindsight this was not a good approach To my defence I had a look at the different classes and tried to find out which features are suitable as modifiers The fact I used features with a current type as result type was because of the expanded classes All the features doing an operation were included with this approach For example all the operations like infix or infix of the class INTEGER have as a result type an integer and modify the state of the object But of course considering the command query separation principle only features with no return type should modify the state of the object Therefore all features that have a return type cannot be a modifier The second approach was to admit all features under the feature clauses Element change and Basic operations The problem of this approach was that the Test Wizard should be language independent Furthermore there are no rules how the feature clauses have to be named The final approach is that all commands are allowed as modifiers 1 all features without a return type except the fe
83. ftware and the documentation The result may be a partial implementation of the objectives without implying any penalty on the success of the project Quality of software Use of Design by Contract o Routine pre and postconditions o Class invariants o Loop variants and invariants e Careful design o Design patterns o Extendibility o Reusability o Careful abstraction Test Wizard Master Thesis Core principles of OOSC2 9 o Command query separation o Simple interfaces o Uniform access o Information hiding o Etc Style guidelines Correct and robust code Readability of the source code Ease of use Quality of documentation Completeness Understandable documentation Usefulness Structure METHOD OF WORK The technologies involved are Gobo Eiffel Lint 3 Programming language Eiffel 7 EiffelStudio 5 4 QUALITY MANAGEMENT Quality was ensured by Weekly progress reports to the supervisor Detailed progress reports for each milestone Review of each milestone by the supervisor concluded by a meeting see validation steps below An intermediary report Documentation see documentation below DOCUMENTATION e Progress reports Short weekly reports and detailed reports for each milestone will describe the progress and encountered difficulties to the supervisor 10 Developer manual This manual documents the software architecture and its limitations describes the difficulties encountered during the imple
84. g runtime in the last two cases Figure 27 shows two example feature calls The first is a catcall and the second feature call works correctly The feature equal of class ANY has the following signature equal some ANY other like some BOOLEAN The first call of equal fails because the feature 75 equal of class INTEGER 16 REF is used The second call works fine because this time the feature of class ANY is used any 1 any 2 ANY int 16 INTEGER 16 REF int INTEGER bool BOOLEAN any 2 int 16 bool int equal any 2 any 1 bool int equal 1 2 Figure 27 Catcall The catcall checking is done neither in features of expanded classes nor in features of classes that do not inherit the feature conforms to of class ANY This feature is used to check if the types are compatible The feature is not exported to the classes FILE NAME DIRECTORY NAME and BOOL STRING of the EiffelBase library Another problem I have encountered are infinite loops and C routines An example of infinite loops is given in Chapter 5 4 The problem of features that call a C routine is that they cause runtime panic if they fail therefore it exists no possibility to rescue from the failure The Test Wizard cannot deal with these two problems Features that allocates memory are also likely to cause runtime panic if they are tested automatically I have used two different strategies to prevent that features can allocate too big chun
85. han this class Generic classes with constraints were also an issue A generic derivation is only valid if the chosen actual generic parameter conforms to the constraint For example if we want to replace an object corresponding to the deferred class PRIORITY QUEUE G gt PART COMPARABLE by the object corresponding to LINKED PRIORITY QUEUE G gt COMPARABLE it is not possible The type LINKED PRIORITY QUEUE PART COMPARABLE simply does not exist because the actual generic parameter does not conform to the constraint see Figure 26 Figure 26 Constrained generic classes 6 4 TEST Pointer handling was also a problem Using a pointer that was generated respectively to generate a pointer is likely to fail Therefore no pointer type is used to 61 Test Wizard Master Thesis generate the pool of objects The user has to provide a pointer if he wants to test a feature or to use a creation procedure with a pointer type as argument This is done in the same way than creating a user defined bound Another problem is the file handling The generation is also likely to fail but this is not yet considered To avoid runtime panic caused by a catcall the type information is checked at run time A catcall is possible if the feature uses an anchored type as arguments The anchor can be an attribute or function of the enclosing class an argument of the enclosing feature or Current To avoid catcalls the type information can be checked durin
86. he tool also allows partial checking of the assertions The user can enable or disable the checking of preconditions postconditions class invariants loop invariants and variants and check instruction to adjust his needs o Testing order The user can choose between performing tests on one feature at a time performing tests on one class at a time or testing as many features as possible which means calling all requested features once before performing further calls which is the default policy The information handler generates a first test scenario from these user defined parameters and passes it to the test handler The test handler outputs the generated scenario and gives the user the opportunity to modify it before the executable gets created The test handler enables the user to modify o Callorderings o List of calls o Arguments of the actual calls Generating a test executable The code generator generates Eiffel code corresponding to the defined test scenario and calls the Eiffel compiler which builds the actual test executable The test executable which is launched automatically contains the necessary code to create the context the pool of test objects Once the context handler has established the order and means to instantiate the needed classes the context generator actually generates the objects and calls modifiers randomly on those objects to build the context Then the call simulator performs the required calls Stori
87. his feature simply adds the tw object but only if the object is not already included 25 Test Wizard Master Thesis The feature add feature to context see Figure 6 adds to the context all tw objects the feature needs These tw objects are the objects representing the class and the arguments First the arguments that are of type TW TYPE are represented by the class TW OBJECT i e the attribute object of class TW TYPE is set Then the attribute class object of the class TW FEATURE is set This tw object represents the class the feature belongs to Now the class object is added to the context of course only if it is not already included Then the argument objects are added if the feature is in test This query is in test was set when creating the scope Furthermore if a feature is under test and the feature has as argument a class with a formal generic parameter replaced by an actual generic parameter then this class is added with the actual generic parameter to the context as well For example the feature fill of the class STRING has the following signature fill other CONTAINER CHARACTER Therefore the tw object representing the type CONTAINER CHARACTER is also added to the context Finally if a feature under test has as an argument a deferred class then the descendant classes are added to the context otherwise the base class of the argument is added For example the feature fill the class CONTAINER ANY is not added but the descen
88. ical representation as well as files using the Gobo Eiffel Test format 4 66 Test Wizard Master Thesis 8 USER MANUAL 8 1 INSTALLATION To install the Test Wizard please follow the instructions below Download the package test wizard zip The package is composed of the directories librarybase and test wizard The first directory contains the file librarybase ace which is the input of the tool The second directory contains the directories tw analyzer tw ast tw code generator information handler and test wizard Moreover it includes the ace file of the Test Wizard called test wizard ace e Unzip the package in a directory of your choice on your disk and set the environment variable STEST WIZARD to that directory Set Compiler path and Finish freezing path in the class TW CODE GENERATOR appropriately These path names will be replaced by relative path names in the next version of the Test Wizard Download the package gobo33 zip and follow the installation instructions Change the postcondition of feature base actual of class TYPE to base type actual not void Result Void definition Result same named type base type context universe actual parameters type i context root context context root context a universe named type named Result is named type and change postcondition of feature base type actual of class E
89. iled INTEGER 16 REF clone failed INTEGER 16 REF standard clone failed INTEGER 16 REF standard twin failed INTEGER 16 REF deep clone failed INTEGER 16 REF deep copy failed INTEGER 16 REF setup failed INTEGER 16 REF passed 1 INTEGER 16 REF tagged out failed INTEGER 16 REF print failed INTEGER 16 REF Operating environment passed 1 INTEGER 16 REF default rescue failed INTEGER 16 REF default create failed INTEGER 16 REF do nothing failed INTEGER 16 REF default failed INTEGER 16 REF default pointer failed INTEGER 16 REF Void passed 1 INTEGER 16 REF infix lt failed INTEGER 16 REF infix gt failed INTEGER 16 REF infix gt failed INTEGER 16 REF max failed INTEGER 16 REF min failed INTEGER 8 REF passed 1 INTEGER 8 REF code failed INTEGER 8 sign failed INTEGER 8 REF failed INTEGER 8 REF zero failed INTEGER 8 REF char failed INTEGER 8 REF Min value passed 1 INTEGER 8 REF value passed 1 INTEGER 8 REF infix lt failed INTEGER 8 is equal failed INTEGER 8 way comparison passed 1 INTEGER 8 item failed INTEGER 8 REF divisible failed INTEGER 8 exponentiable failed INTEGER 8 failed INTEGER 8 is_valid_character_code failed INTEG
90. infix feature and generate feature respectively generate deferred prefix feature generate deferred infix feature and generate deferred feature are called to generate the code for the feature call These features can be found under the feature clause Feature generation The feature names of prefix features contain the word prefix and the feature names of infix features contain the word infix therefore it is not a problem to differentiate the different kinds of features The feature generate feature is the most complex one because it can have the highest number of arguments The features to generate infix or prefix features have to handle only one respectively no argument at all The feature generate feature relies on the feature search argument object that looks for the right argument object and places it in the attribute ast object found item furthermore it also sets the attribute actual parameters The feature as well as the attributes are in the class CODE GENERATOR SUPPORT because they are also needed to generate the test feature calls The feature has three arguments The first is the argument for which the object is searched the second is the object on which the feature is called and the third argument is the descendant in the case that the object 15 of a deferred class otherwise it is just again the object itself Therefore the feature can be used for both to find argument objects of deferred and effective classes A deferred
91. ing type failed 7 0 3 0 0 INTEGER 8 REF conforms to failed 9 0 1 0 0 INTEGER 8 REF type failed 6 0 4 0 0 INTEGER 8 consistent failed 3 0 7 0 0 INTEGER 8 standard is equal failed 4 0 6 0 0 INTEGER 8 REF equal failed 1 0 3 0 6 INTEGER 8 standard equal failed 5 0 1 0 4 INTEGER 8 REF equal failed 3 0 3 0 4 INTEGER 8 REF failed 4 2 4 0 0 INTEGER 8 REF standard copy failed 5 1 4 0 0 INTEGER 8 REF failed 8 0 2 0 0 INTEGER 8 REF standard clone failed 5 0 5 0 0 INTEGER 8 standard twin failed 5 0 5 0 0 INTEGER 8 REF deep clone failed 5 0 5 0 0 INTEGER 8 REF deep copy failed 3 2 5 0 0 INTEGER 8 REF setup failed 5 0 5 0 0 INTEGER 8 passed 0 0 0 0 0 INTEGER 8 REF tagged out failed 7 0 3 0 0 INTEGER 8 REF print failed 7 0 3 0 0 INTEGER 8 REF Operating environment passed 0 0 0 0 0 INTEGER 8 REF default rescue failed 7 0 3 0 0 INTEGER 8 default create failed 5 0 5 0 0 INTEGER 8 REF nothin failed 6 0 4 0 0 INTEGER 8 REF default failed 5 0 5 0 0 INTEGER 8 default pointer failed 7 0 3 0 0 INTEGER 8 REF Void passed 0 0 0 0 0 INTEGER 8 infix lt failed 5 1 4 0 0 INTEGER 8 REF infix gt failed 4 1 5 0 0 INTEGER 8 REF infix gt failed 3 3 4 0 0 INTEGER 8 REF failed 4 2 4 0 0 INTEGER 8 REF failed 5 0 5 0 0 85 9 2 2 CLUSTER KERNEL CLASSIC Test Wizard Master Thesis
92. ion exception a file put string assertion violation 44 Test Wizard Master Thesis end else set other exception a file put string other exception violation end a file put string if tag name void then a file put string tag name end a file put string N failure failure 1 retry end feature Test feature test feature a context TW CONTEXT is deferred end feature Generate exceptions generate exceptions a file PLAIN TEXT FILE is Generate results in a file require do end not void a file void a file put string associated class name a file put string feature name T if assertion violation exception 0 or other exception 0 or possible catcall 0 then a file put string failed elseif no exception 0 then a file put string passed elseif precondition violation exception 0 then a file put string no call was valid else a file put string could not be tested end a file put string 5T a file put integer no exception a file put string T a file put integer precondition violation exception a file put string T a file put integer assertion violation exception a file put string T a file put integer other exception a file put string a file put integer possible catcall a file put string T a file put integer void target a file put string N generate exceptions xml a fi
93. ion violation valid increment CHARACTER REF infix precondition violation valid increment CHARACTER REF infix precondition violation valid increment CHARACTER REF infix precondition violation valid decrement CHARACTER REF infix precondition violation valid decrement CHARACTER REF infix precondition violation valid decrement CHARACTER REF infix precondition violation valid decrement CHARACTER REF infix gt precondition violation other exists CHARACTER REF infix gt precondition violation other exists CHARACTER REF max precondition violation other exists CHARACTER REF min precondition violation other exists CHARACTER REF min precondition violation other exists CHARACTER REF min precondition violation other exists CHARACTER REF conforms to precondition violation other not void CHARACTER REF same type precondition violation other not void CHARACTER REF copy precondition violation other not void Table 8 The file details generated testing the class CHARACTER REF 72 Test Wizard Master Thesis class feature result no precondition assertion other possible void target exception violation violation exception catcall object CHARACTER item passed 10 0 0 0 0 0 CHARACTER REF code assed 10 0 0 0 0 0
94. ks of memory The first strategy was to add preconditions to the features that allocate memory I have done that with the features make resized area and aliased resized area of class SPECIAL This new class SPECIAL can be found in the directory extra library Naturally the generated project is compiled again with this new class The problem 15 that not only the class SPECIAL allocates memory It is not only cumbersome to add the preconditions to all these classes but it also impacts 62 Test Wizard Master Thesis portability across compilers which do not have the same base classes The second strategy was to allow only integer values between 1000 and 1000 The drawback is that the class INTEGER cannot be tested properly probably also some other classes Besides a user can also allocate as much memory as he wants to 63 64 Test Wizard Master Thesis Test Wizard Master Thesis 7 OUTLOOK I will discuss in this chapter the possible improvements of the Test Wizard A graphical user interface as well as a database should be added to the next version of the Test Wizard The Test Wizard is not completely user friendly at the moment It would be nice to gather the user information through a graphical interface A database would enable to store the test scenario for regression testing Both the information about the test executable as well as the test results could also be stored Furthermore the compiler path as well as the pa
95. lass CHARACTER_REF in XML format 9 1 Test Wizard Master Thesis 9 APPENDIX CLUSTER AND CLASSES OF THE LIBRARY EIFFELBASE 1 kernel classic l ANY 2 TUPLE 3 STRING 4 ARRAY 5 BOOLEAN 6 CHARACTER 7 INTEGER 8 INTEGER 8 9 INTEGER 16 10 INTEGER 64 11 REAL 12 DOUBLE 13 POINTER 14 DOUBLE_REF 15 REAL_REF 16 PLATFORM 17 MEMORY 18 SPECIAL 19 FUNCTION 20 PLAIN_TEXT_FILE 21 UNIX_SIGNALS 22 GC_INFO 23 FILE 24 POINTER REF 25 ROUTINE 26 CHARACTER REF 27 BOOLEAN REF 28 ARGUMENTS 29 ISE RUNTIME 30 MEM INFO 31 RAW FILE 32 STREAM 33 DECLARATOR 34 DIRECTORY NAME 35 PATH NAME 36 STORABLE 37 EXCEPTIONS 38 UNIX FILE INFO 39 BASIC ROUTINES 40 PROCEDURE 41 DIRECTORY 42 FILE NAME 43 TYPED POINTER 2 no cluster GENERAL 2 NONE 3 BIT 3 kernel INTEGER REF 2 NUMERIC 3 COMPARABLE 4 HASHABLE 5 INTEGER INTERVAL 6 PART COMPARABLE 7 MEM CONST 8 IO MEDIUM 9 STRING HANDLER 10 CONST 11 INTEGER 64 REF 12 TO SPECIAL 3 SEQ STRING 14 STD FILES 15 INTEGER 16 REF 16 INTEGER 8 REF 4 list SEQUENCE 2 ARRAYED LIST 3 LIST 4 BI LINKABLE 5 LINKABLE 6 LINKED LIST 7 SORTED TWO 8 CHAIN 9 DYNAMIC LIST 10 TWO WAY LIST 11 TWO WAY CIRCULAR 12 LINKED WAY LIST CIRCULAR 3 SORTED LIST 14 FIXED LIST 15 DYNAMIC CIRCULAR 16 CIRCULAR 17 DYNAMIC CHAIN 18 ARRAYED _CIRCULAR 9 PART_SORTED 20 C
96. lds the context a pool of objects The main features of this class are generate container class and generate context class GENERATION OF CLASS TW CONTAINER The first feature creates the file tw container e containing the class TW CONTAINER see Figure 10 It represents lists implemented by resizable arrays with the additional feature purge duplicates that allows to purge duplicates in the list The class TW CONTAINER is used as a container holding the objects of the context After the context initialization the feature purge duplicates is called on every container to delete duplicates The effect of this implementation choice is less control over the number of objects that the user has chosen by setting the randomness level Therefore the randomness level just defines the number of creations but not the final number of objects in the context This implementation choice is justifiable because after all having an object twice or even more times in the context is useless The feature purge duplicates has a rescue clause The reason of the rescue clause will be explained in Chapter 5 1 3l Test Wizard Master Thesis indexing description Test Wizard Automatic generated file representing the container used by the context date Date 2004 01 20 21 33 54 class TW CONTAINER G inherit ARRAYED LIST G rename copy as al copy is equalas al is equal select al al is equal end EXCEPTIONS create make m
97. le PLAIN TEXT FILE is Generate results in xml file a file require do not void a file void a file put string T lt feature gt N if feature name has or feature name has 4 then a file put string T T lt name gt lt CDATA feature name gt lt name gt N else a file put string lt gt feature name lt name gt N 45 Test Wizard Master Thesis end a file put string T T lt class gt associated class name lt class gt N if assertion violation exception gt 0 or other exception gt 0 or possible catcall gt 0 then a file put string T T lt result gt failed lt result gt N elseif no exception 0 then a file put string T T lt result gt passed lt result gt N elseif precondition violation exception 0 then a file put string T T lt result gt no call was valid lt result gt N else a file put string T T lt result gt could not be tested lt result gt N end a file put string 7 lt exception a file put integer no exception a file put string no exception 96N file put string VS T9eT precondition violation a file put integer precondition violation exception a file put string lt precondition_violation gt N a file put string T T lt assertion_violation gt a file put integer assertion violation exception a file put string assertion violation 96N a file put string T T
98. le degree 4 a universe compile degree 3 create analyzer make a universe analyzer analyze all create info handler make analyzer structure a cluster info handler cluster implementation 1 a class info handler class implementation 26 a cluster info handler add class a class info handler add class to context a class info handler add bound CHARACTER REF character m info handler add bound CHARACTER REF character info handler set randomness level 10 info handler set testing order 3 create code generator make info handler code generator generate container class code generator generate context class code generator generate test class code generator generate all classes code generator generate root class code generator generate ace file code generator compile end Figure 28 Feature analyze universe of class TW TEST WIZARD 68 Test Wizard Master Thesis The commands available to set the test parameters are described in the following section You can set the scope of the test with the features listed in the following figure add cluster a cluster TW CLUSTERH add class a class TW CLASS add features a feature list ARRAYED LIST TW add feature a feature TW FEATURE Figure 29 Features to set the scope of the test These features need either an object of type CLUSTER TW CLASS TW FEATURE You can retrieve the clusters classes and features using th
99. library 2 Defining the test scenarios Then the tool gathers user information to define the test scenarios It involves two parts the information handler and the test scenario handler The information handler receives the system information and enables the user to choose the following test criteria Scope of the test The user can select a cluster to be tested which means all classes of the cluster are tested The user can also make a more fine grained selection and choose some classes only or even just a few features o Exhaustiveness The tool enables changing the number of calls to be performed at several levels globally per cluster per class and per feature Moreover the user can control if a feature is tested in descendants Test Wizard Master Thesis o Context The tool lets the user decide how classes will be instantiated Predefined bounds are used for common types such as basic types besides the user can define bounds for every type The user can also select which creation procedure the tool uses to instantiate classes overriding the default procedure the tool would choose Furthermore the user can give the level of randomness of targets and parameters which fixes the number of variants to be created o Tolerance The user can define if a test has passed or not in the case that a rescued exception occurred during a test execution By default all assertions are checked on the tested features Nevertheless t
100. lt other_exception gt a file put integer other exception a file put string other exception 96N file put string T T lt possible_catcall gt a file put integer possible catcall a file put string lt possible_catcall gt N a file put string T T lt void_target gt a file put integer void target a file put string void target 96N a file put string lt gt end feature Name access associated class name STRING Class name feature name STRING Feature name exhaustiveness INTEGER Number of calls feature Name setting set class name a name STRING is Set associated class nameto a name require name not void a name void name not empty not a name is empty do associated class name a name ensure name set associated class name is equal a name end set feature name a name STRING is Set feature nameto a name require name not void a name void name not empty not a name is empty do feature name a name ensure name set feature name is equal a name 46 Test Wizard Master Thesis end set exhaustiveness a int INTEGER is Set exhaustiveness to a int require not negative a int gt 0 do exhaustiveness a int ensure exhaustiveness set exhaustiveness a int end feature Exceptions access no exception INTEGER Number of calls with no exceptions precondition violation exception INTEGER
101. mask passed 10 0 0 0 0 INTEGER REF out passed 10 0 0 0 0 INTEGER REF generator passed 10 0 0 0 0 INTEGER REF generating type passed 10 0 0 0 0 INTEGER REF conforms to passed 8 2 0 0 0 INTEGER REF same type passed 10 0 0 0 0 INTEGER REF consistent passed 10 0 0 0 0 INTEGER REF standard is equal passed 5 5 0 0 0 INTEGER REF equal failed 3 0 0 0 7 INTEGER REF standard equal failed 2 0 0 0 8 INTEGER REF deep equal failed 4 0 0 0 6 INTEGER REF copy passed 8 2 0 0 0 INTEGER REF standard copy passed 9 1 0 0 0 INTEGER REF clone passed 10 0 0 0 0 INTEGER REF standard clone passed 10 0 0 0 0 INTEGER REF standard twin passed 10 0 0 0 0 INTEGER REF deep clone passed 10 0 0 0 0 INTEGER REF deep copy passed 9 1 0 0 0 INTEGER REF setup passed 10 0 0 0 0 INTEGER REF io passed 10 0 0 0 0 INTEGER REF tagged out passed 10 0 0 0 0 INTEGER REF print passed 10 0 0 0 0 INTEGER REF Operating environment passed 10 0 0 0 0 INTEGER REF default rescue passed 10 0 0 0 0 INTEGER REF default create passed 10 0 0 0 0 INTEGER REF do nothin assed 10 0 0 0 0 INTEGER REF default passed 10 0 0 0 0 INTEGER REF default pointer passed 10 0 0 0 0 INTEGER REF Void passed 10 0 0 0 0 INTEGER REF infix lt passed 8 2 0 0 0 INTEGER REF infix gt passed 8 2 0 0 0 INTEGER REF infix gt passed 9 1 0 0 0 INTEGER REF max passed 9 1 0 0 0 INTEGER REF min passed 8 2 0 0 0 INTEGER 64 REF item passed 10 0 0 0 0 INTEGER 64 REF hash code passed 10 0 0 0 0 INTEGER 64 REF sign pa
102. mentation and explains how to extend the current version User manual The user manual describes the usage of the Test Wizard Test Wizard Master Thesis e Intermediary report The intermediary report consists of the intermediary Thesis report developer manual The thesis report consists of the final developer manual the final user manual and a theoretical part discussing the profiling VALIDATION STEPS The validation of each milestone comprises Report Sending detailed report and the relevant parts of the work to the supervisor for review e Meeting Organizing a meeting with the supervisor for presentation and discussion of the conducted work Revision Revision of parts or all of the work for this milestone depending on the conclusion of the supervisor 2 1 5 PLAN WITH MILESTONES PROJECT STEPS Milestones Objectives MI Software architecture Design of the entire software architecture M2 Eiffel analyzer Implementation M3 Information handler Implementation M4 Code Generator Implementation M5 Context handler and context generator Implementation M6 Call simulator Implementation M7 Test result handler Implementation M8 Demo application Implementation M9 Intermediary report M10 Database storing Implementation Mil Database query handler Implementation M12 Test scenario handler Implementation M13 Developer manual M14 User manual M15 Thesis report
103. name of the inherited class generate create also has a string representing the creation procedure name generate feature clause takes as argument a category this string must start with Finally the feature generate end simply creates the end with no further arguments These features rely on the feature put string of class PLAIN TEXT FILE 30 Test Wizard Master Thesis generate indexing a file PLAIN TEXT FILE a description STRING generate class a file PLAIN TEXT FILE a deferred BOOLEAN a class name STRING generate inherit a file PLAIN TEXT FILE a class name STRING generate create a file PLAIN TEXT FILE a creation procedure name STRING generate feature clause a file PLAIN TEXT FILE a feature clause STRING generate end a file PLAIN TEXT FILE Figure 9 Features of class TW CODE GENERATOR SUPPORT generating general parts of a class The features of the parameter generation are also placed in this class because both TW CONTEXT GENERATOR and TW TEST GENERATOR need these features For the same reason the features under the feature clauses Status report Status setting and Search are placed in this class These features will be explained later The class CODE GENERATOR CONSTANTS contains the constants and keywords used in the generated code Logically they can be found under the feature clauses Constants and Keywords 4 3 2 CLASS TW CONTEXT GENERATOR The context generator bui
104. ng results into a database Running the test executable and storing the results into a database to handle regression testing Test Wizard Master Thesis 5 Outputting test results Displaying the results to the user The results are classified into four categories o Passed o Could not be tested o No call was valid o Failed The test result handler provides the user with result information in the following formats A graphical representation o XML files o Hiles using Gobo Eiffel Test format 4 Test Wizard Master Thesis Eiffel analyzer System information Contract eguipped Eiffel library GUI Test GUI Test Test Information scenario scenario handler handler V Code Test GUI Context Feature call Test generator executable fader generator simulator result EN er query result handler Inout Output Gabo Eiffel Test format Storage Figure 1 Architecture of the Test Wizard 1 1 Gather system information 2 Display system information to the user and gather user information 3 Build the test scenario from the criteria selected by the user 4 Store the test scenario into a database for regression testing 5 Generate a test executable corresponding to the test scenario 6 Run the executable it creates a pool of objects the Context possibly helped b
105. nt from the container creation of effective classes The command generate deferred container creation 18 called The difference is that the container creation of objects of deferred classes creates containers for the descendants of the deferred class but not for the class itself An important query is is constraint conform to check if the actual generic parameters conforms under any constraint to the formal generic parameters of the descendant This is the case if the class of the actual generic parameter is a descendant of the class of the formal generic parameter of the descendant class see Chapter 6 3 3 The next two parts the creation of void objects and the creation of an object of type ANY are just done by writing one line of code for each It just calls the features create void object and create object The generation of these features is explained later The following part the creation of predefined default bounds also calls only the feature create xxx object xxx stands for the class name including the generic parameters Every object in the context is checked by the query is special type so that only the needed predefined default bounds are created An object is a special type if it is either a boolean character or number type see the first column of Table 2 The object initialization includes the object creation and modification The following code piece see Figure 14 is generated for every object in the context
106. ocedure only with another one in case the creation procedure fails a few times As mentioned before there are still features used as modifiers that do not change the state at all To avoid this the code of the features could be analyzed to see which feature really alters the state of the objects Furthermore it would be nice to give to the user the possibility to exclude a feature as modifier Even a list of features that are inconvenient modifiers could be stored in the database naturally only when a database is added to the Test Wizard but this 15 not the case at the moment 65 Test Wizard Master Thesis 7 2 TEST The Test Wizard does not test features of obsolete classes but it does test obsolete features An improvement would be that the user can select if he wants to test obsolete classes and or obsolete features The user cannot choose the actual generic parameters Only the type ANY is used to instantiate generic classes It would be nice if other types are used as actual generic parameters or even if the user can choose the actual generic parameters that he wants to use Catcall checking could be added for the case that the anchor is an attribute or function of the enclosing class In the current version of the Test Wizard the type information is only checked at run time if the anchor is an argument of the enclosing feature or Current Furthermore expected output formats for future releases of the Test Wizard are a graph
107. ome if some void or other void or other conforms to some then Figure 21 Catcall checking To generate the code for the real test feature call the features generate test generate infix test generate prefix test respectively generate test catcall generate infix test catcall and generate prefix test catcall These features are written using the features from feature clause Feature generation of class TW CONTEXT CLASS as a model therefore they are not explained in detail 4 3 4 CLASS TW CODE GENERATOR The main features of class TW CODE GENERATOR are generate ace file and generate root class As the name suggests the first feature generates the ace file of the generated project The tolerance that the user has chosen is put in action It is a test criteria defined by the information handler The ace file has a default part containing the option section The option assertion sets the assertion level of the Eiffel system Table 7 shows the different assertion levels option assertion level check check instructions require preconditions ensure postconditions loop loop invariants and variants invariant class invariants Table 7 Assertion levels The list of clusters in the ace file contains of course the two directories tw_code and test the generated files are placed in these directories The feature generate root class relies on the feature gene
108. on of the classes described in chapter 3 and explains the implementation choices 4 1 EIFFEL ANALYZER The class ANALYZER implements the Eiffel analyzer The input is an ET UNIVERSE created by gelint The universe is created in the root class TW TEST WIZARD root procedure parses the ace file The needed scanner and parser to parse the ace file is generated using gelex respectively geyacc The parser generates an AST abstract syntax tree representing the constituants of an Eiffel class The root procedure sets the error handler so that only errors and warnings are reported and it calls the feature analyze universe This feature sets the compiler to the ISE Eiffel compiler Then the features set use xxx keyword are called They specify if xxx is considered as a keyword or just as an identifier The tool sets assign attribute convert and reference as keywords These four keywords are not part of the current version of Eiffel but they will be included in the next revision of the language The next commands see Figure 4 called are equivalent to the different compilation degrees The command parse all corresponds to the compilation degrees 6 and 5 a universe parse all a universe compile degree 4 a universe compile degree 3 Figure 4 Gelint compilation degrees The main class feature analyze all puts the required system information in the structure ARRAYED LIST CLUSTER Let s have a closer look at the five cla
109. onally in the attribute default creation procedure of type TW FEATURE The descendants of the class can be found under the attribute descendants of type ARRAYED LIST CLASS The queries is deferred and is expanded give information about whether it is a deferred or an expanded class CLASS TW FEATURE The required feature information is the feature name the class the cluster name the arguments and the result type of the feature If the feature is a creation procedure or a modifier or if it is deferred this 1s also part of the required feature information The queries is creation procedure is modifier and is deferred are at disposal to check this information The feature and the cluster name are stored in the attributes name and cluster both of type STRING whereas the class is stored in associated class which is of type TW CLASS The arguments and the result type are stored as the names suggest in the attributes arguments and result type The type of the first attribute 15 ARRAYED LIST TW TYPE and the second 15 TYPE CLASS TW TYPE The class is used for the feature s arguments and result type It contains the class name as well as the class and the actual generic parameters Furthermore the query is like type gives information if it is an anchored type The class name and the class are stored in the attributes name and associated class of type STRING respectively TW CLASS The actual generic parameter if it exists
110. onsidered as passed or not passed in the case that an exception occurred during the test execution and was rescued However this is not handled in the current version of the Test Wizard set check instructions set no check instructions set preconditions set no preconditions set postconditions set no postconditions set loop invariants set no loop invariants set loop variants set no loop variants set class invariants no class invariants set rescue allowed set no rescue allowed Figure 34 Features to enable and disable assertion checking The tool offers three different testing orders You can set the testing order using the feature set testing order a order INTEGER The different testing orders have the following meaning e Testing order 1 Performing tests on as many features as possible e Testing order 2 Performing tests one feature at a time e Testing order 3 Performing tests one class at a time 71 Test Wizard Master Thesis The demo application sets the testing order to the second testing order see Figure 28 therefore the features are tested one after the other 8 3 LAUNCH THE GENERATED PROJECT Once you have set all parameters you can launch the tool The Test Wizard generates another project and compiles it Unfortunately it is not possible to run the generated project within the Test Wizard yet Therefore you have to run the generated program which is generated in the directory
111. quence of this is that if you try to copy 0 into an INTEGER int copy 0 you get a precondition violation on other Void which proves that the parameter value is completely misinterpreted To sum up this is a real bug very well hidden and I m very glad the wizard found it To avoid this bug expanded types have to be explicitly converted into the corresponding reference types before passing it to features with an anchored type as argument The Test Wizard does not convert the expanded types therefore base types cannot be tested Reference types can be tested In the appendix you can find the results of the test of classes BOOLEAN REF CHARACTER REF REAL REF DOUBLE REF INTEGER REF with the default testing order The default testing order tests each feature once and starts then from the beginning continuing the testing The following two pop up windows in Figure 23 can appear when testing the feature infix These errors are not visible in the Test Wizard because calling this feature do not cause an exception in case of a domain or overflow error Only these pop up windows appear in this case x C pow DOMAIN error 63 pow OVERFLOW error Figure 23 Power operation errors 5 3 NOT TESTABLE CLASSES The Test Wizard cannot test the class POINTER As a consequence all classes depending on the class POINTER cannot be tested as well unless the user provides some obj
112. r passed 10 0 0 0 0 TUPLE uniform integer 32 passed 10 0 0 0 0 TUPLE is uniform integer 64 passed 10 0 0 0 0 TUPLE is uniform pointer passed 10 0 0 0 0 TUPLE is uniform real passed 10 0 0 0 0 TUPLE is uniform reference passed 10 0 0 0 0 TUPLE convertible to double passed 10 0 0 0 0 TUPLE convertible to real passed 10 0 0 0 0 TUPLE arrayed passed 10 0 0 0 0 TUPLE boolean_arrayed passed 10 0 0 0 0 TUPLE character_arrayed passed 10 0 0 0 0 TUPLE double_arrayed passed 10 0 0 0 0 TUPLE integer_arrayed passed 10 0 0 0 0 TUPLE pointer_arrayed passed 10 0 0 0 0 TUPLE arrayed passed 10 0 0 0 0 TUPLE string arrayed passed 10 0 0 0 0 TUPLE correct mismatch failed 0 0 0 10 0 TUPLE is hashable passed 10 0 0 0 0 TUPLE generator passed 10 0 0 0 0 TUPLE generating type passed 10 0 0 0 0 TUPLE conforms to passed 10 0 0 0 0 TUPLE same type passed 10 0 0 0 0 TUPLE consistent passed 10 0 0 0 0 TUPLE is equal passed 3 7 0 0 0 TUPLE standard is equal passed 5 5 0 0 0 TUPLE equal failed 2 0 0 0 8 TUPLE standard equal failed 3 0 0 0 7 TUPLE deep equal failed 1 0 0 0 9 TUPLE passed 6 4 0 0 0 TUPLE standard copy passed 5 5 0 0 0 TUPLE clone passed 10 0 0 0 0 TUPLE standard clone passed 10 0 0 0 0 TUPLE standard twin passed 10 0 0 0 0 TUPLE deep clone passed 10 0 0 0 0 TUPLE deep copy passed 2 8 0 0 0 TUPLE setup passed 10 0 0 0 0 TUPLE io passed 10 0 0 0 0
113. rate processing It generates the file tw root containing the root class feature generate processing generates the feature execute 50 Test Wizard Master Thesis The feature execute first initializes the context calling the feature make Then all the generated classes for the test feature calls are put in the container fest container of type ARRAY TEST The different testing orders are taken into account generating the feature execute The feature are tested by calling the feature run on a class in the container est container The feature execute generates three different files details txt file which is an Excel compatible text file containing each error occurred during the test execution and the files exceptions txt and exceptions xml These files contain the same namely the summary of all errors and the test results Whereas the first file is the same format as the details txt file the other is an XML file This three files are generated using the feature put string of class PLAIN FILE The different testing orders are taken into account with the features generate feature testing generate class testing and generate all testing The first feature generates the feature execute so that a feature is tested at a time The second generates it in such a way that a class 15 tested at a time The last feature which is the default testing order generates the tests in that way that as much feature
114. reate context file make context file name create added objects make 0 a directory current working directory change working directory test directory name name context sort fill extra scope context generate indexing context file 9o Test Wizard Automatic generated file containing the context context file open append generate class context file False context class name generate create context file make generate feature clause context file Initialization generate initialization generate feature clause context file Creation generate all creation generate feature clause context file Modification generate all modification generate feature clause context file Special Bounds generate all bounds generate feature clause context file User defined code bound file open read from bound file start until bound file end of file loop bound file read line context file put string bound file last string context file put new line end context file put new line bound file close generate feature clause context file Container generate all container generate feature clause context file Access generate all access generate end context file change working directory a directory end Figure 11 Feature generate context class of class TW CONTEXT GENERATOR The next step establishes the order in which objects will be instantiated see Figure 11 The commands name context and sort
115. respectively for the descendants in the case of an object of a deferred class The randomness level specifies how many objects of a certain type are created Furthermore the program does not modify any object in a container if the container does not have at least two objects The containers are first filled with a Void object and therefore if the container contains only one object it is for sure Void These is the reason for the integer 2 39 Test Wizard Master Thesis if objects count lt randomness level then create xxx object end if objects count gt 2 then modify xxx object end Figure 14 Object initialization An exception are the objects of type POINTER as well as objects of a special type They are not initialized in this part These generated code pieces are in a loop that loops randomness level 5 The next part is the user defined bound creation It checks for every object in the context if the attribute bounds is not empty This attribute is set by the information handler in the case the user has added a bound If this attribute is not empty the code line create xxx bounds is generated The next line of generated code contains the feature call create any descendants These features are explained later The final part of the creation procedure are the feature calls purge duplicates for every container It deletes duplicates in the container This is the end of the initialization part of the
116. rgument set object temp object a argument object end Figure 15 First code piece of feature search argument object If the object found is of type ANY and the actual generic parameters of the deferred object are not empty then the object is replaced by the actual generic parameter see Figure 16 if temp object associated class is equal ANY and not a object actual generic parameters is empty then a object actual generic parameters first set object search object a object actual generic parameters first object temp object last object found item end Figure 16 Second code piece of feature search argument object The next step is to replace objects declared of deferred types by an object declared of an effective type feature search effective class sets the attribute effective class found item feature is implemented class TW CODE GENERATOR SUPPORT In the case that the object of the deferred type has actual generic parameters they are kept in the attribute actual parameters The feature search effective class handles a special case The deferred class ROUTINE should not be replaced by the class FUNCTION because this class has one formal generic parameter more the result type That is why the class ROUTINE 18 always replaced by the class PROCEDURE In addition the feature takes care that the formal generic parameters of the effective class found conform to the constraint 42 Test Wiz
117. rom the class name also the generic parameters so that the names are unambiguous Table 5 contains a few examples of container and object names Type Container name Object name INTEGER integer objects context integer or non void context integer ARRAY CHARACTER array character objects context array character or non void context array character PROCEDURE ANY procedure any tuple op any cp objects context procedure any tuple op any TUPLE ANY non void context Table 5 Container and objects names The feature type list name has only one generic parameter list as input therefore it can easily generate a string of this parameter list The feature parameter name has two arguments The first argument is the actual generic parameter list and the second is the formal generic parameter list If both the formal generic and the actual generic parameter lists are not empty then the feature calls type list name with the actual generic parameter list as argument In the other case if the actual generic parameter list is empty then the feature is called with the formal generic parameter list Finally the feature deferred parameter name has two arguments like the feature parameter name But instead of choosing one of the generic parameter lists this feature merges the two generic parameter lists together The feature checks for every formal generic parameter if an actual generic parameter exists r
118. rted It is not only a creation procedure but also a feature The export status of the features is not stored because only features that are exported to ANY are further processed Furthermore the class FEATURE has the attribute is creation procedure to know if the feature is a creation procedure To differentiate the creation procedure from the feature it is stored twice of course only if the export status allows it If not even one creation procedure of a class is exported to ANY then the class is excluded and not handled at all The Test Wizard could not instantiate an object of that base class in this case The Test Wizard chooses a default creation procedure that will be used to create objects of that type to generate the context Naturally only if the user did not change the creation procedure The selection criterion of the Test Wizard is simple It just takes the first creation procedure provided that the feature does not have the current or pointer type as an argument A creation procedure with a pointer as argument is almost sure to fail A feature with current type as argument would not produce anything interesting and it is even likely that it fails too For example the creation procedure make from array from the class ARRAY has the argument a declared as ARRAY The only object that can be passed as an argument is Void because that is the only object in the container before the creation part The feature call would therefore
119. s set globally a number INTEGER set cluster a number INTEGER a cluster TW CLUSTER set class a number INTEGER a class TW CLASS set feature number INTEGER a feature TW FEATURE set descendants a feature TW FEATURE Figure 31 Features to set the exhaustiveness You can set the context by calling the features in Figure 32 For the moment it is required to have both the context and the scope set to the same clusters classes or features Setting a context that is different than the scope could be interesting when a database is added to the Test Wizard but this is not the case at the moment add cluster to context a cluster TW CLUSTER add class to context a class TW CLASS add feature to context feature TW FEATURE set creation procedure a name STRING a creation procedure TW FEATURE set randomness level a level INTEGER add bound a name STRING a bound ANY Figure 32 Features to set the context You can also change the creation procedure the Test Wizard chooses by default for a specific class The feature set creation procedure sets the creation procedure using as arguments the class name and an object of type FEATURE Furthermore you can set the randomness level which specifies how many times a feature under test is called Finally you can add self defined bounds You have to provide a feature for each bound written in the text file bounds which has to be situated in th
120. s as possible are tested This means that each feature is tested once before calling a feature a second time 51 52 Test Wizard Master Thesis Test Wizard Master Thesis 5 TEST OF EIFFELBASE 5 1 THE FEATURE PURGE DUPLICATES In chapter 4 3 2 it is mentioned that the feature purge duplicates has a rescue clause In fact this rescue clause has the same reason as all the other rescue clauses in the class TW CONTEXT They should prevent the program to fail during the context generation It is more important to get a context at all than to get exactly the context we wanted In fact the context is created randomly anyway Therefore a container that contains an object twice is better than no container at all The feature purge duplicates failed twice while testing the EiffelBase library The problems were the containers of the INTEGER 8 REF and INTEGER 16 REF objects both holding the object initialized with the Min value The minimal value is 128 respectively 32768 Calling the feature caused the invariant sign times abs saying that an integer is equal to its sign multiplied by its absolute value to be violated This invariant is wrong in the case of these minimal values since the absolute value cannot be coded on this number of bits For example the absolute value of 128 15 128 with these number of bits as well instead of 128 Multiplying 128 with 1 15 0 and not 128 therefore the invariant is violated 5 2 BASE TYPES
121. ss or only a feature The first step is done in the information handler initializing the context The algorithm used can be described as follows e Adding objects instances of the classes that the user wants to test They are either all the classes of a cluster if the user wants to test a cluster the class if he wants to test a class or the classes containing the features if he wants to test only a few features e Adding objects instances of the classes that depend on the objects added at the first step They are all the classes used as arguments in the creation features the modifiers and in the test features and the descendants classes of an argument or the enclosing class of a test feature if the class is deferred To avoid an infinite loop the structure added context classes is introduced It is checked if a class is already added to the context before actually adding it into the context In the code generator the objects are sorted so that an object that depends on another object i e the creation procedure has the other object as argument is created later This is not really necessary because the creation and modification of the objects is placed in a loop If an object fails to be instantiated in the first loop iteration it is probably instantiated in the following loop iteration because the containers are holding more objects at every loop iteration The information handler does not care about the generic parameters in a s
122. ssed 10 0 0 0 0 81 Test Wizard Master Thesis INTEGER 64 REF one passed 10 0 0 0 0 INTEGER 64 REF zero passed 10 0 0 0 0 INTEGER 64 REF ascii char passed 4 6 0 0 0 INTEGER 64 REF Min value passed 10 0 0 0 0 INTEGER 64 REF Max value passed 10 0 0 0 0 INTEGER 64 REF infix lt passed 10 0 0 0 0 INTEGER 64 REF is equal passed 8 2 0 0 0 INTEGER 64 REF three way comparison passed 10 0 0 0 0 INTEGER 64 REF set item passed 10 0 0 0 0 INTEGER 64 REF divisible passed 7 3 0 0 0 INTEGER 64 REF exponentiable passed 9 1 0 0 0 INTEGER 64 REF is hashable passed 10 0 0 0 0 INTEGER 64 REF valid character code passed 10 0 0 0 0 INTEGER 64 REF abs passed 10 0 0 0 0 INTEGER 64 REF infix passed 10 0 0 0 0 INTEGER 64 REF infix passed 9 1 0 0 0 INTEGER 64 REF infix passed 8 2 0 0 0 INTEGER 64 REF infix passed 8 2 0 0 0 INTEGER 64 REF prefix passed 10 0 0 0 0 INTEGER 64 REF prefix passed 10 0 0 0 0 INTEGER 64 REF infix passed 8 2 0 0 0 INTEGER 64 REF infix passed 8 2 0 0 0 INTEGER 64 REF infix passed 10 0 0 0 0 INTEGER 64 REF to boolean passed 10 0 0 0 0 INTEGER 64 REF to integer 8 passed 7 3 0 0 0 INTEGER 64 REF to integer 16 passed
123. sses that are implemented to store the system information and see how the required information is stored 4 1 1 SYSTEM INFORMATION STORAGE As the name suggests an object of type TW CLUSTER represents a cluster holding the required system information of a cluster and so on I will call these objects in future tw cluster not to cause a confusion between them and real clusters This is also valid for the objects created from the other classes 21 Test Wizard Master Thesis CLASS TW CLUSTER The cluster information is stored in the class CLUSTER It contains the cluster name and the parent cluster name stored in the attributes name and parent both of type STRING as well as all classes of the cluster stored in the attribute classes of type ARRAYED LIST CLASS CLASS TW CLASS The class information consists of the class name and the name of the cluster it belongs to the feature the class contains the generic parameters if the class has any and all descendants of the class It is also important to know if the class is deferred or expanded The class and the cluster name can be found in the attributes name and cluster both of type STRING features of the class are stored in the attribute features of type ARRAYED LIST TW FEATURE The modifiers are all features of the class without a return type namely procedures They are stored in the attribute modifiers of the same type The default creation procedure is stored additi
124. t lt possible_catcall gt O lt possible_catcall gt lt void_target gt 0 lt void_target gt lt feature gt lt feature gt lt name gt code lt name gt lt class gt CHARACTER_REF lt class gt result passed lt result gt no gt 10 lt exception precondition violation20 precondition violation assertion violation20 assertion violation other exception20 other exception possible catcall20 possible catcall void target20 void target lt feature gt lt feature gt lt name gt default_pointer lt name gt lt class gt CHARACTER_REF lt class gt lt result gt passed lt result gt lt no_exception gt 10 lt no_exception gt lt precondition_violation gt 0 lt precondition_violation gt lt assertion_violation gt 0 lt assertion_violation gt lt other_exception gt 0 lt other_exception gt lt possible_catcall gt 0 lt possible_catcall gt lt void_target gt 0 lt void_target gt lt feature gt lt feature gt lt name gt Void lt name gt lt class gt CHARACTER_REF lt class gt result passed lt result gt no gt 10 lt exception precondition violation20 precondition violation assertion violation20 assertion violation other exception 0 other exception possible catcall20 possible catcall void target20 void target lt feature gt lt test_results gt Figure 35 Extract of the test results of testing the c
125. t Wizard which generates test cases automatically from contract equipped classes 1 Besides the goal of the Test Wizard was to serve as a workbench to try out different testing strategies Test Wizard Master Thesis Test Wizard Master Thesis 2 PROJECT PLAN 2 1 PROJECT DESCRIPTION The project description contains the objectives and priorities before starting the project and do not reflect what was actually done Not everything was realized like described in the project description Chapter 3 informs about the differences between the actual work and the project plan This project description is a long term picture of what a final version of the Test Wizard should look like 2 1 4 SCOPE OF THE WORK Contracts are a solid information basis to generate black box test cases automatically The Test Wizard takes an Eiffel library as input and generates test cases using the contract information see Figure 1 Architecture of the Test Wizard 1 The test bed I used was the Eiffel library for fundamental structures and algorithms EiffelBase 12 8D The library specification which is expressed by assertions is the key to generate black box test cases automatically The Test Wizard follows a five step process 1 Gathering system information The Eiffel analyzer parses the library EiffelBase provided as input to retrieve the system information The system information contains in particular the list of clusters classes and features in the
126. tem associated class from a class descendants start until a class descendants after loop if a class formal generic parameters count a class descendants item formal generic parameters count then add class to context a class descendants item a class descendants finish end a class descendants forth end end arguments forth end elseif a feature is creation procedure or a feature is modifier then from arguments start until arguments after loop if not is pointer arguments item name then arguments item set object add to context arguments item object add class to context arguments item associated class end arguments forth end end end Figure 6 Feature add feature to context of class TW INFORMATION HANDLER The next upper level is the feature add class to context It adds all objects a class needs to the context For efficiency the attribute added context classes of type ARRAYED LIST CLASS is introduced This structure contains all tw classes that are already added to the context because it is senseless to check if every tw object of a tw class are already added to the context given that the tw class is already added Therefore the feature first checks if the tw class is already added if this is the case the feature does nothing If the tw class is not already added it goes through all features calling the feature add feature 10 context In addition all descendants of the tw class are added if the query in
127. th to the finish freezing exe executable used for freezing Eiffel code are stored with the full path in the attributes compiler path and finish freezing path in class TW CODE GENERATOR These paths should be replaced by relative path names Above all the generated project is compiled by the code generator but not automatically run yet 7 1 CONTEXT The exception handling in the context is not really advanced In fact the rescue clause is only to prevent the program to fail and the exceptions are not handled A finer exception handling already in the context file would be nice The catcall checking is only done in the test feature calls but not when generating the context It could be added there as well The catcall checking in the context would be part of an advanced exception handling mechanism I encountered difficulties with catcalls most of the time using features of class ANY These features are anyway excluded from the context generation 7 1 1 CREATION PROCEDURES AND MODIFIERS The Test Wizard could consider more than one creation procedure The Test Wizard sets a default creation procedure in the current version In fact the user has the possibility to change the default creation procedure to another one but the context uses only one creation procedure even if the creation procedure is changed For example the creation procedures could be called randomly like the modifiers Another approach would be to replace a creation pr
128. the current object To avoid catcalls the type information is checked during run time in the last two cases First of all it is checked if the feature has an anchored type as argument using the query include_like_argument which relies on the attribute is_like_type defined in class TW_TYPE If the feature has an anchored type as argument it is checked with the query is_real_catcall if it concerns one of the two cases that can be checked Furthermore features of expanded classes are excluded as well as features of classes that do not inherit the feature conforms_to of class ANY This feature is used to check if the types are compatible This feature is not exported to the classes FILE_NAME DIRECTORY_NAME and BOOL_STRING of the EiffelBase library set_arguments a_feature create temp_feature make conforms to a feature associated class a feature cluster if include like argument and is real catcall and not a feature associated class is expanded and a feature associated class has feature temp feature then else end Figure 20 Generation of test feature 49 Test Wizard Master Thesis The feature generate catcall generates the code with catcall checking This feature relies on catcall checking that generates the code for every anchored type argument The generated code checks the type compatibility for these types see Figure 21 Feature signature Generated check code deep equal some ANY other like s
129. the scope if at least one feature of the class is going to be tested The object is not handled if the class is not part of the scope If the argument type of a test feature is deferred then it is replaced by an effective type in the call But the feature search_argument_object which is explained later searches first the deferred object in the context and replaces it afterwards with an effective one Therefore the object has to be in the context but is not needed to generate the context from context start until context after loop if not context item is deferred then else if is in test context item then end end context forth end Figure 12 Context loop schema 37 Test Wizard Master Thesis The second part takes care of creating the containers I will start with the implementation description of the container creation of objects of effective classes The command generate container creation is called in this case It allocates a container of type ZW CONTAINER with zero items for every object The command relies on the feature parameter name This feature is used for the naming in the class CONTEXT Three features namely deferred parameter name parameter name and type list name deal with naming They be found class TW CODE GENERATOR SUPPORT under the feature clause Parameter generation The name used in class TW CONTEXT to represent the objects or the object containers includes apart f
130. ting the class CHARACTER REF 72 Table 9 Test results of testing the class CHARACTER REF see 73 Table 10 Indexes of clusters and 55 ene 76 LIST OF FIGURES Figure 1 Architecture of the Test Wizard 1 6 Figure 2 Actual architecture of the Test Wizard eei 14 Figure 3 The class TW CODE GENERATOR and its ancestors 17 Figure 4 21 Figure 5 Extract of feature analyze features of class TW 24 Figure 6 Feature add feature to context of class TW INFORMATION 27 Figure 7 Feature is in test of class TW INFORMATION 28 Figure 8 An example file of user defined bounds 28 Figure 9 Features of class TW CODE GENERATOR SUPPORT generating general PAS OL E du ME s ane ie NU ES 31 Figure 10 Class TW CONTAINER pen te RS ete M Lue ein 22 Figure 11 Feature generate context class of class TW CONTEXT GENERATOR 34 Figure 12 Context loop schema uuo ine luci R 37 Figure 13 Code extract from feature type list string of class IW CODE GENERATOR 39 Figure 14 Object initialization t 40 Figure 15 First code piece of feature search argument object
131. trict way therefore a few objects used as arguments are not in the context that is passed to the code generator The code generator introduces an additional context These additional objects are added to the context before generating the files First the feature fill_extra_scope_context loops through all the features in the scope In the case that the argument object is a generic class with an actual generic parameter it is added to the extra context Before the extra context is added to the context it is checked again if they are not already included Then it calls the feature fill_extra_scope_context This feature loops through the context and looks at the arguments of all creation procedures and modifiers Afterwards the extra context is also added to the context like in the feature fill_extra_scope_context The feature loops again over the context until no more objects are added to the extra context It is really important that the objects declared of deferred types are added to the context too because the feature search_argument_object looks in the case of a deferred class for the object declared of the corresponding deferred type and replaces the class afterwards with an object of an effective class 58 Test Wizard Master Thesis 6 3 1 CREATION PROCEDURES A creation procedure has two export statuses The first one describes to which classes the creation feature is exported and the second one describes to which classes the feature is expo
132. tw code This directory contains the root class and the ace file besides the compiled program the other files of the program are generated in the directory tw test This directory contains a class for each feature under test as well as the class TW CONTEXT and the deferred class TW TEST The generated program generates three different files Two Excel compatible text files called details txt see Table 8 and exceptions txt see Table 9 and an XML file called exceptions xml see Figure 35 The generated program writes a line into the file details txt each time a test feature call did not work The written line contains the class and the feature name the kind of exception and the associated tag for assertion violations The file exceptions is provided in two different formats They are generated after the test feature calls They contain the summary of all the exceptions as well as the result The possible results are passed e could not be tested e no call was valid e failed class feature kind tag CHARACTER_REF is equal precondition violation other not void CHARACTER REF is equal precondition violation other not void CHARACTER REF three way comparison precondition violation other exists CHARACTER REF infix precondition violation valid increment CHARACTER REF infix precondition violation valid increment CHARACTER REF infix precondit
133. y also added to the tw class if is creation procedure a feature then if temp feature clients has class any class then a class add feature a feature end if is creation procedure exported to any a feature then a creation procedure a feature standard twin a creation procedure set creation procedure a class add feature a creation procedure end else a class add feature a feature end Figure 5 Extract of feature analyze features of class TW ANALYZER The last feature called is analyze descendants It adds the descendants to each tw class using the feature convert descendants This feature is placed at the end because it does not create new tw classes instead it looks for the tw classes in the structure and just adds them as descendants therefore there exists only one object for each class required system information can be accessed by the structure ARRAYED LIST CLUSTER 24 Test Wizard Master Thesis 4 2 INFORMATION HANDLER The class TW INFORMATION HANDLER implements the information handler The class takes as input the structure that contains the system information created by the Eiffel analyzer The different test criteria are ordered by the feature clauses Scope Exhaustiveness Context Tolerance and Testing order 4 2 1 SCOPE The attribute scope contains all features that should be tested There are four features at disposal to add features to the scope The feature add cluster adds
134. y the user 7 Store the order of class instantiations for regression testing 8 The executable performs feature calls on the pool of instantiated objects 9 Store the test results into a database 10 Output the results to the users 11 Display the results graphically with diagrams 12 Generate XML files corresponding to the test results 13 Generate files using the Gobo Eiffel Test format 14 Query the database and retrieve test results to be passed to the test result handler Test Wizard Master Thesis 2 1 2 INTENDED RESULTS Implementation Eiffel analyzer Information handler Test scenario handler Code generator Context handler Context generator Call simulator Test result handler Database query handler Database storing Demo application The Eiffel analyzer uses the tool Gobo Eiffel Lint gelint 2 3 to have access to the system information required by the information handler and makes the required system information available to the information handler The information handler interacts with a GUI where the user can create a test scenario by feeding test parameters The current implementation done during this project does not include any GUI development This is the long term picture of the Test Wizard The test scenario handler also interacts with a GUI where the user can adapt the generated test scenario This is again the long term picture of the Test Wizard Th

Final report - Chair of Software Engineering

Contents

Download Pdf Manuals

Related Search

Related Contents