User Manual
Contents
1. As shown in Figure 5 each mutant has its own folder in which it saves all necessary informa tion All mutants generated will be stored in the path muScriptHome sessionName result Classname traditional_mutants pe OOOO GO sessionl result cal gt traditional_mutants gt int_ Organize v Include in library v Share with v Burn Nev AOIS1 AOIS22 ACIS 43 AOIS64 Ji RORAS AOI2 AOI23 AOI 44 JAOIS65 J ROR16 AOS3 JAOI24 aoas AOI 66 J ROR17 J AOIS_4 AOIS25 AOI 46 AOIS67 J ROR18 QAOI5 AOI 26 AOIS47 AOIS68 ROR19 J aose PAOI27 AOI48 Ji AoIs69 J ROR20 J AOIS_7 J aos 28 J Aaors49 J AOIS70 J ROR21 Jaossa aors_29 AOIS5O J ROR1 Ji ROR22 AO9 J aos30 AoIs51 J ROR2 Ji ROR23 AOI10 J aos31 AOIS52 Ji ROR3 J ROR 24 AOIS11 J aos32 AOIS53 Ji ROR4 J ROR_25 B aos12 J aos33 AOIS5S4 J RORS5 iJi ROR26 AOIS13 J aos34 J aos55S J ROR6 J ROR27 J aosi4 J aos35 AOIS5SG J ROR7 ROR28 B aosis J aos36 AOISST J ROR8 j ROR29 J aosi6 J Aaor37 J Aois58 J ROR9 J ROR30 J aos17 aos38 J AoIs59 J ROR10 Ji ROR31 J aosi J aors39 J AoIs60 J ROR11 J ROR32 J aos19 J Aors40 J AOIS61 J ROR12 J ROR33 B aos 20 Jj aos41 J Aos 62 J ROR13 J ROR 34 J aos 21 J Aaos42 J AoIs63 J ROR14 104 items Figure 5 ROR and AOIS Mutants in the result Folder 2 all lt session name gt The all option enables all 15 traditional method level mutation operators as shown in Figure 6 Thus the2. and by the markequiv command result_list csv can be opened in Excel or other spreadsheet applications and mutant_list can be opened in any text editor runmutes can run in one of three modes if default Each test is run against mutants that are alive not killed or marked equivalent This is the typical way mutation testing is used in practice dead Each test is run against all live and dead mutants but not equivalent mutants This supports experimental studies An additional option equiv can be added to run the tests against all equivalent mutants as well fresh A new results file with a time stamp is generated The old results file is not used thus all mutants start as alive and all tests are run against all live mutants The default dead and fresh modes are mutually exclusive Parameters lt operator namel gt lt operator name2 gt lt testset name gt lt session name gt The lt operator name gt is one of AORB AORS AOIU AOIS AODU AODS ROR COR COD COI SOR LOR LOI LOD ASRS ALL The operator names can be either lower or upper case If no operator name is provided by default wScript will run tests against all mutants This is equivalent to using the name ALL The lt session name gt is an already existing test session 2Tip Make sure JUnit is correctly configured in the system environment Ze The lt testset name gt is the name of a JUnit te
3. cal java class has 173 mutants Also if the current session has more than one class genmutes will generate mutants for all classes C gt mujava java mujava cli genmutes ALL sessioni 1 cal java File CG mujava sessionisrcecal java Total mutants gnerated for cal java 1735 Figure 6 Generating ALL mutants 5 Creating Tests uScript uses JUnit tests that are supplied by the tester They can be created in any IDE or command line tools and must be compiled to class files outside of uScript As shown in Figure 7 the class files need to be put in the folder session name testset which is created by the testnew command The tester can add new JUnit test files at any time Thus a common process might be to create a few tests run the mutants on the tests then examine the live mutants and either mark them as equivalent or generate additional tests This process can repeat until the tester is satisfied with the mutation score In this way the mutants become guides to help the tester design good tests Y session1 gt I result Y J testset calTest4 class calTest2 class calTest3 class calTest class gt O classes gt O src Figure 7 JUnit Test Files in the testset Folder 6 Running Tests Command runmutes The program runmutes runs tests against mutants It saves results in two files in the result folder named result_list csu and mutant_list The result files are used in additional calls to runmutes
4. be Users username mujava 2 Debug mode optional Debug_mode We can set the debug_mode to either true or false Enabling debug_mode will display more detailed intermediate results on the console 3 Classpath To get Script running six library jar files must be defined correctly in the classpath mujava jar commons io jar openjava jar tools jar gyunit jar and hamcrest core jar For example on a Windows platform if all the jar files are stored in the directory c mujava extend CLASSPATH in the Environment Variables by adding the paths c mujava junit jar c mujava hamcrest core 1 3 jar c mujava mujava jar c mujava openjava jar c mujava commons io 2 4 jar on a Unix Linux platform if all the jar files are stored in the directory home Ideng2 mujava the following command will define them in the classpath export CLASSPATH CLASSPATH home ldeng2 mujava mujava jar home ldeng2 mujava commons io 2 4 jar home ldeng2 mujava openjava jar usr local java jdk1 7 0 lib tools jar home ldeng2 mujava junit jar home ldeng2 mujava hamcrest core jar 3 Creating a Test Session Command testnew A test session defines an independent mutation test on a program component It comprises source files in Java bytecode class files JUnit tests and results These are stored in four directories that testnew creates testset result classes and src Figure 1 shows the direc
5. cessfully CoNmujavarjava mujava cli testnew sessiond c nujavassre cal Source file is compiled successfully Session is built successfully CoMWMuUjJavasjava mujava cli testnew session4 cisSmujavassre cal java Source file is compiled successfully Session is built successfully Figure 2 Example Commands for Creating New Test Sessions Java source files should usually be saved in the src folder of the home directory so that only file names are required when creating a new session the first example command in Figure 2 However uScript is also able to fetch source files from other directories with an absolute path such as shown in the last two commands in Figure 2 2 lt debug gt If the configuration file does not set the debug mode it can be set as an option to testnew Options given on the command line have higher priority than the configuration file so they override the settings in the file Note the debug option is available with ALL subsequent commands An example is shown in Figure 3 4 Generating Mutants Command genmutes l Prerequisite Make sure the current version of Openjava jar from the pJava website is correctly set in the system s environment Conmujavasjava mujava cli testnew debug sessioni cal Make GisNmujava ssessioni directory Making Gismujava ssessionl directory done Make GCismujavassessionil sspre directory Making Glsmujavassessionissre directory done Make Gismujava ssession
6. esult Files Cmutant_list and result_list csv gt for details Use markequiv command to mark equivalent mutants Figure 9 Running Tests Against 50 of the Mutants Randomly Chosen 7 Marking Mutants as Equivalent Command markequiv The program markequiv is used to mark live mutants as equivalent Parameters l lt class name gt lt mutant 1 gt lt mutant 2 gt lt mutant n gt lt session name gt The lt class name gt specifies the name of the target class The options lt mutant 1 gt lt mutant 2 gt specifies names of live mutants as listed in result_lust csv The lt session name gt takes the session name The example in Figure 10 marks the mutant named ROR_8 of class cal in session1 as equiv alent If a mutant is already dead pScript will skip it if a mutant does not exist it will be ignored C mujava java mujava cli markequiy cal HOR_ sessioni All equivalent mutants are marked Figure 10 Marking an Equivalent Mutant
7. i classes directory Making GClismujavassessionil sclasses directory done Make GCoNmujava ssessioni result directory Making Closmujavassessionl sresult directory done Make Gismujava ssessioni testset directory Making Cismujava ssessionl testset directory done Source file is compiled successfully Session is built successfully Figure 3 Creating a New Test Session With the Debug Mode Conmujavarjava mujava cli genmutes R R AQIS sessioni 1 cal java File Cis mujava sessioni sre cal java Total mutants gnerated for cal java 164 Figure 4 Generating ROR and AOIS Mutants The program genmutes is used to generate mutants The current version is only able to generate method level mutants Parameters L lt operator name 1 gt lt operator name 2 gt lt session name gt Each lt operator name gt is one of AORB AORS AOIU AOIS AODU AODS ROR COR COD COI SOR LOR LOI LOD ASRS ALL The operator names can be either lower or upper case If no lt operator name gt is provided by default uScript will generate mutants for all operators This is equivalent to using the name ALL The lt session name gt takes the name of a test session that already exists For example Figure 4 shows a command to generate all ROR and AOIS mutants for the class in sessionl Note that pJava discards mutants that do not compile uScript prints the total number of mutants generated
8. muscript User Manual Feb 2014 musScript User Manual 1 Introduction musScript Script is a command line version of the muJava wJava mutation analysis tool It uses uJava s core mutation engine The purpose of pScript is to facilitate experimental studies using mutation testing By combining a few flexible commands users can design scripts run a large amount of tests and collect execution results without using GUI interface of pJava As with pJava wScript is provided on an as is basis We have no funds and very limited resources to provide support or even respond to queries uScript has four commands as listed in Table 1 They are all included in the pJava jar file and called with the syntax java mujava cli command Table 1 uScript Commands Command Description testhew Create a test session Run test cases against mutants Mark mutants as equivalent a 2 Setting up the uScript Configuration File Before running psScript it is necessary to set the correct directory location and other configuration information The configuration file is named mujava config It is an extended version of the pJava configuration file with the possibility of adding additional options 1 Home directory location MuJava HOME An absolute path is required so that Script can find classes under test find test sets to save results For example in Windows we can set it as c mujava in a Mac Linux or Unix machine it could
9. st file without the class extension If no testset name is given uScript will run all test sets in the testset directory Figure 8 shows the test set calTest being executed against all mutants in the session sesszonJ After execution the total number of mutants killed the number of total mutants and the mutation score are displayed Conmujavarjava mujava cli runmutes calTest sessioni Class Mame cal Test Mame calTest Running Current running mode default Total mutants killed 8H Total mutants 184 Mutation Score 4 7692 Please look at the result Files Cmutant_list and result_list csv for details Use markequiv command to mark equivalent mutants Figure 8 Running Mutants in default Mode all lt testset name gt lt target session name gt The all option runs the test set against all 15 of the traditional method level mutation operators p lt random percentage gt Sometimes we would like to randomly choose a subset of the total mutants so the p option is provided This is a common experimental technique We can specify a number from 0 to 1 as the random percentage For example Figure 9 uses p 0 5 to run tests against 50 of the live mutants chosen randomly CioNmujavarjava mujava cli runmutes p 6 5 calTest sessioni Class Mame cal Test Mame calTest Running Current running mode default Total mutants killed 42 Total mutants 52 Mutation Score 4 8077 Please look at the r
10. tories created by testnew for the session session Computer Local Disk C mujava sessionl Include in library Share with Burn New Name ds testset result aces dk classes wit src eami Figure 1 Test Session Directory The program testnew also prepares for subsequent processes such as compiling Java source files and copying compiled class files into the new session created Parameters l lt session name gt lt src file name 1 gt lt src file name 2 gt lt src file name n gt The lt session name gt will be used as the name of the new session and the root directory of the test session If a session with that name already exists in the home directory of pScript an error message will be returned and the program will exit without making any changes The source file names can be given with or without the java extension Normally one test session contains only one class under test however it is possible to put multiple classes in one session This is normally done when the class under test uses another class Figure 2 shows four example commands that can be used to create a new test session for the Java source code named cal java saved in c mujava sre folder Conmujavarjava mujava cli testnew sessioni cal Source File is compiled successfully Session is built successfully C gt mujava java mujava cli testnew sessions cal java Source file is compiled successfully Session is built suc
Download Pdf Manuals
Related Search