Mika User Manual Version 1.2 Midoan Software Engineering
Contents
1. GUI Area 5 Progress Warnings and errors are displayed in this area vi GUI Area 6 File View This area contains a simple file viewer to view but not edit to minimise the risk of corrupting production code source code and generated tests The generated tests are contained in a file named lt file name_no_extension gt mika_tests txt in the current working directory 7 The Tools Back End Mika can also be used directly via the command line This does not provide new functionalities as the GUI front end offers all the functionalities of the back end tools but allows users to integrate Mika in their favourite environment e g GPS or to use Mika in batch mode There are two tools provided mika_ada_parser the parser tool andmika_ada generator the test data generator tool i The mika ada parser Parser Tool Page 9 of 13 The parser tool mika_ada_parser parses the indicated file and all dependent files to produce an output suitable for the subsequent test data generation phase handled by the mika_ada_generator tool Note that if the file under test or any of its dependences is not up to date mika_ada_parser will compile it prior to the parsing phase proper You can avoid this behaviour by always ensuring that the file under test and all its dependences are up to date prior to calling mika_ada_parser As explained below you can control this behaviour by providing a GNAT project file via the e switch and a target working2. directory via the w switch The basic syntax note that the double quotes are actually necessary on the command line is mika_ada_parser M lt full path _of install dir gt gnat83 gnat95 gnat05 optional _ switches lt file name no extension gt the order of the switches is not significant M lt full path of install dir gt is compulsory and must provide the full path to the bin directory of Mika file e g M C Program Files Midoan Software Engineering Solutions Mika bin one of the Ada version switch i e gnat83 or gnat95 or gnat05 must be provided even if a GNAT project file is used to indicate the Ada standard of the file under test the last argument must be the name of the file under test without extension e g oilandgasmonitoring gasflowcalibrationcheck Optional Switches f lt full path to gnat bin gt provides the full path to bin directory of the gnat version you wish to use e g F GNAT 2009 bin If itis not specified the gnat version reachable via your PATH environment variable is used o lt full path of initial file gt provides the full path to the file under test or if it is used the full path of the project file e g o F vv70 code If it is not specified the current directory is used instead w lt full path of target dir gt specifies a directory for the parser s entire output If this switch is o
3. rns Te eer terre 4 AS AUS EE iad pe ee cous uacev sawn aa cunt e E a 4 SUM ION aaa 5 A WaT dicn a aaea a e b aAa e 5 MaS LIMITATIONS A A RARA 5 cele INIA A A dean te eueecauubaduacuee avenues 6 A SUT eo Ui EM Barter a O E ee E RN 6 LGUI Area 1 Drive and Folder CaS iii A ii 6 1 Introduction Mika is a new generation testing tool in fact we believe that it is the first tool of its kind for industrial code written in a professional programming language Mika generates tests that will achieve when executed the highest possible branch decision or MC DC coverage of your code Mika does not generate a large number of tests nor any random tests to achieve its aim each test is carefully constructed to exercise a previously uncovered decision branch or MC DC in the code under test Mika offers for the first time the possibility to automatically generate test data to achieve maximum possible coverage of the source code under test the only necessary input is the original source code There are no annotations no testing scripts to write Mika can generate under ten minutes a small set of tests from subprograms that may lead to the execution of 100 000s of line of code The tests generated will be aimed at achieving the highest possible code coverage of the subprogram under test and of all the subprogram it itself calls Mika works for a substantial subset of Ada that largely encompasses the SPARK Ada subset if you hav
4. Mika User Manual Version 1 2 Midoan Software Engineering Solutions Ltd Copyright 2010 Midoan Software Engineering Solutions Ltd Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one Permission is granted to copy and distribute translations of this manual into another language under the above conditions for modified versions except that this permission notice may be stated ina translation approved by Midoan Software Engineering Solutions Ltd Abstract Mika is an automatic test data generation tool for code written in a subset of Ada 83 Ada 95 or Ada 2005 Mika uses genetic algorithms in its attempt to generate inter subprograms tests that will by construction exercise during execution the maximum possible number of branches decisions or MC DCs in the code under test Keywords automatic test data generation unit testing integration testing Ada Spark Ada Table of Contents A acc a A E so ade aaaleGueniay 2 EE ATE o rae eee ee eae ee 3 A O tent uanah uaa anuLgbausaceuces 4 CONT ek ee EE Renee eee ere tre er Peasy etre Per ere eee re eee rere ee ene ne ere eee eee emer
5. cters 7 Intrinsic subprograms are not all accurately handled These accuracy limitations imply that on occasions the predicted coverage by Mika will not be achieved during actual execution that the expected outcome will differ or that the optimal coverage will not be achieved iii Miscellaneous Limitations Currently include 1 Subprograms local to another subprogram cannot be directly tested 6 The GUI Front End Mika can be used via the simple GUI front end or as detailed in the next section via the command line for easier integration with your favourite IDE The GUI is divided into 6 main areas Mika by Midoan Full Version 1 2 on 28 June 2010 11 31 10 File Options Help F DO NOT USE E SEE yenes ch8_3 adb Tests test_array_aqqregates Tests test_array_aggregates ts DFN ch8_3 test_array_aggregates APS Es bck_Mika Path F bck_Mika workin examples Parse Generate Tests E che Ada Version Coverage Strategy O Ada 83 9 Branck Decision MCDC MIKA TEST DATA GENERATOR a Copyright Midoan Software Engineering Solutions Ltd Oe 3 PALLET ESAS 4 poms ei aldean coil ada 05 Level of Coverage LE Subprogram only v Directory F bck_Mika examples ch8 Project switches Package chB_3 Subprogram test_array_aggregates Strategy decision Context not ignored L _ V Time Stamped Directory Coverage Depth everything including the RTL Additional gnatbind switches Time Stamp 05 05 2010 21 07 48 Post Gene
6. directory within the working directory whose name is time stamped The current working directory can be changed via Options gt Set Working Directory The time stamped directory is thus unique and takes the form lt subprogram_ name gt year month day hour minutes seconds otherwise the directory is just lt subprogram name gt and previous tests will be lost Note that if the subprogram under test is an operator the lt subprogram_ name gt is known internally as E string lt ascii_code gt E g willbeknownas string 42 and and will be known as string 97 110 100 The Compile and Run Tests option compiles and runs the automatically generated test driver To avail of this option users must manually Mika is never allowed to automatically change user s code insert test points in the specification and body of the packages under test These tests point must be of the form procedure Mika Test Point Test number in Integer in the specification part And procedure Mika Test Point Test number in Integer is separate in the body part A file named packageName mika test point adb must be added in the source folder Mika never compromises the integrity of the original source folder it always works in its own working directory never writing anything nor modifying anything in the original source folder This test point file must be of the form separate packageName procedure Mika Test Point Test numb
7. e SPARK Ada like code Mika does not rely on annotations Mika should be able to automatically generate targeted test data using the source code as is Mika has been designed to leave your original code and the containing folders untouched your code will not be modified in any way during the test data generation process Mika has been designed to be as simple as possible to use pick the file containing the Ada subprogram you wish to generate tests for pick the subprogram and click the generate tests button You should be able to evaluate the suitability of Mika for your own circumstances in minutes Mika does not aim to replace traditional well established coverage testing tools its main functionality is the automatic creation of purpose built test data that should by construction achieve the highest possible level of coverage of your source code It replaces a manual process These tests can serve as input into traditional tools to certify the level of testing coverage achieved Mika replaces the tedious expensive time consuming error prone and usually incomplete process of manual test data creation Mika generates tests that exercise called subprograms as thoroughly as the caller subprogram tests are not just unit tests they can be used for integration testing purposes Mika is not a static analysis tool no false positive are ever generated The tests it produces are complete and directly executable Mika provides at no extra cos
8. end section in either cases the lt subprogram_name gt will subsequently be known internally in its ASCII form especially for directory creation Page 11 of 13 one of the testing strategy switch i e Tbranch Tdecision or Tmcdc must be provided to indicate the test data generation testing criterion desired Refer to the GUI front end section for further details one of the context switch i e Cignored or Cnot_ignored must be provided If the context is not ignored the tests generated will be based according to the outcome of the elaboration phase of the package containing the subprogram under test If the context is ignored the effects of the elaboration phase will be ignored Refer to the GUI front end section for further details the last argument lt file name no extension gt must be the name of the file under test without extension e g oi landgasmonitoring gasflowcalibrationcheck It can also be elaboration in which case tests will be generated to execute the elaboration phase only Optional Switches the o lt full path _ of parsed file gt switch provides the full path to the parsed version of the file under test as generated by the mika_ada parser tool e g o C tmp alarm mika If itis not specified the current working directory is used 1 lt line number gt is only used to handle overloaded subprograms It must be used to provided the line number of the first occurrence of the subp
9. er in Integer is begin null end Mika Test Point where packageName should be replaced by the actual package name Test points only need to be added if the user wishes to avail of the automatically generated test driver If the elaboration context is ignored many more than just the package containing the subprogram under test may require the manual insertion of a test point In addition the automatic generation of the test driver may need some help to produce Ada code that actually compiles Users can add context clauses to the automatically generated test points by preceding the first lines of their own corresponding test point with MIKA E g saving MIKA with System use System separate Hardware procedure Mika Test Point Test number in Integer is begin null end Mika Test Point inhardware mika_test point adb in the current working directory will ensure that the automatically test driver will systematically contain the context with System use System In effect the rest of the line following MIKA will be automatically added at the start of the corresponding automatically generated test point Actual test outcomes are automatically compared against expected test outcomes Compilation switches for the test driver can be manually changed or reset The Generate Tests button generates the tests according to the user settings Progress is shown in the area 5 Actual tests are displayed in area 6 v
10. green In any case an already parsed subprogram can be expanded to reveal its subprograms and the elaboration only test data generation option Previous test data generations outcomes for a given subprogram can be further expanded and viewed Once a package file has been selected its containing file is displayed in area 6 iii GUI Area 3 Parsing A package that has not yet been parsed needs parsing Note that the parser will re compile your files if they are out of date prior to its own processing Default switches for the compiler are displayed and can be modified manually For example you can specify the path of the original gnat adc file by adding gnatec C foo gnat adc tothe gnatmake switches box on the GUI do not forget to add it also to the test driver switches box if you wish to compile the auto generated test driver The Reset button uses the settings set via the Options gt Set Default Switches to reset the parse switches for the current package under test The Clear button allows erasing of previous parsing outcomes for the package under tests The Parse button parses the file where the chosen package is to be found Progress is shown in the area 5 The path to the compiler can be set viaOptions gt Set Working Directory iv GUI Area 4 Test Data Generation Once a subprogram has been chosen the test data generation process can be customised Branch Decision or MC DC coverage can be chosen The calling co
11. mitted the current directory is used instead If the directory does not exist it will be created Useful to specify a target working directory e lt GNAT project _switches gt ifa GNAT project file is necessary to compile or browse the source and object files this switch must be used It should contain the normal GNAT switches related to project files P X etc These will be passed automatically to gnatmake gnatbind or gnat ls by the parser Compulsory if a project file is present a lt gnatmake switches gt if gnatmake needs specific switches for the file under test this switch should be used Usually not necessary unless compilation problems occur b lt gnatbind_switches gt if gnatbind needs specific switches for the file under test this switch should be used Usually not necessary unless binding problems occur c lt gnatls_switches gt if gnatls needs specific switches for the file under test this switch should be used Usually not necessary unless browsing problems occur Output mika ada parser generates e aparsed file lt file name_no_extension gt po in the new directory lt full_path_of target _dir gt lt file name no extension gt mika whichis used by the subsequent test data generation phase e asubprogram file lt file name no extension gt subprograms in the directory lt full path _of target_dir gt which is used by the GUI front end A typical example is mika ada pa
12. ntext of the subprogram can also be ignored by ticking the Ignore Context option in other words the initialisations performed during the elaboration process are ignored or taken into account If the context is taken into account the tests generated will be based according to the outcome of the elaboration phase of the package containing the subprogram under test In other words the elaboration context will be taken into account These lead to more realistic tests but may also lead to a lower level of coverage On the other hand ignoring the elaboration context allows the generation of test data that overwrites initialisations made at elaboration time The test may be less realistic but typically will achieve a higher level of coverage The Level of Coverage desired can be indicated e Subprogram Only Mika will only generate tests to cover the subprogram under consideration Page 7 of 13 e Source File Only Mika will generate tests to cover the subprogram under consideration and also any called subprograms from the same source file e Everything Except RTL Mika will generate tests to cover the subprogram under consideration and also any called user written subprograms e Everything Including RTL Mika will generate tests to cover the subprogram under consideration and also any called subprograms including GNAT s Run Time Library Ifthe Time Stamped Directory option is chosen the test files are created in a new
13. periment with if you do not have access to suitable Ada code Feel free to submit your own code examples to Midoan 9 Feedback and Bug Reports Whether you are entitled to user support under Midoan s maintenance terms or not Midoan welcomes and encourages suggestions for improving Mika Use the Feedback button to access Midoan s feature request web page These may include e enlarging OS and or GNAT version integration e adding Ada features not supported in the current subset e adding finer user control of test data generation process e generating tests for other coverage metrics e g condition MCDC e adding new test output formats for better integration with third party tools This is your opportunity to guide the future development of Mika to suit your particular needs Midoan will do its best to address identified bugs and issue revisions to all customers as per the appropriate license agreement Page 13 of 13
14. ration Additional gnatls switches Compile and Run Tests Additional gnatmake switches TEST NUMBER 1 a Additional gnatmake CONSTRUCTED TEST elaboration test driver switches Es mika test point v128 1 0 1 0 1 0 2 0 2 0 0 2 0 2 0 AS 128 2 2 0 2 0 1 0 1 0 21 0 E test_array_aggregates pa r E r r 7009 08 0 E eA 19 11 2009 08 00 44 7128 3 1 0 1 0 e r20 6 19 11 2009 08 01 2 sisas a128 42 42 42 42 42 42 42 42 42 42 42 19 11 2009 08 02 28 ates 012 11 14 01 11 11 00 11 24 11 10 05 05 2010 21 07 28 a129 1 0 0 0 0 0 0 0 0 0 0 0 Generate a129 2 99 21307 9999 lt r i GUI Area 1 Drive and Folder Choosing This where you may pick the folder containing the code you wish to test Mika searches in the background the contents of the chosen folder for Ada source code containing packages ads and adb files and displays the results in the area 2 ii GUI Area 2 Package Subprogram and Test File Choosing The names of the top most packages contained in the chosen folder are displayed If a package has never been parsed its background appears transparent if it has already been parsed but is out of date i e it probably needs re parsing unless the changes made on the source code have no semantics consequences its background appears blue finally if the package is has already been parsed and is up to date its background appears
15. reasonable time and within the memory at its disposal While Mika can handle very large amounts of linear code 100 000s lines subprogram calls and bounded loops on simple data structures without problems the following aspects of the code under test may have a negative impact on Mika s run time and or make Mika reach its memory limitations 1 Large data structures handling e g array objects with more than 1000 elements It is difficult to offer a more precise picture of Mika complexity limitations Limitations are being removed as new version come on stream e g since version 1 2 input dependent dynamic types no longer have a significant impact on the run time of Mika see the release notes for past improvements Mika has been designed to produce results within the reasonable time of the order of minutes it is not uncommon for the compilation time of the automatically generated test driver to be much larger than the test data generation process time itself ii Accuracy Limitations The following accuracy limitations are part of Mika Page 5 of 13 1 Very large integers are not properly handled 2 Very large floating point numbers are not properly handled 3 Floating point numbers representation is approximate 4 Boolean operators on arrays of Booleans are not accurately handled 5 Overloading of Boolean operators with non Boolean returning operators are not accurately handled 6 Wide Characters are modelled using Chara
16. rogram in the code could be in its specification or its body e g 196 t disables the time stamped directory feature of the output directory Refer to the GUI front end section for further details u lt S F A R gt indicates the level of coverage desired S Subprogram Only F Source File Only A Everything Except RTL R Everything Including RTL E g uF If it is not specified uS is the default Refer to the GUI front end section for further details Output mika_ada_generator generates in lt full_path_of parsed file gt lt subprogram_ name gt year month day hour min utes seconds orjustin lt full path of parsed _file gt lt subprogram name gt if the time stamped directory feature was switched off via the t switch e atest results file lt file name no extension gt mika tests txt which contains the automatically generated tests and predicted behaviour of the code e anumber of Ada files composing the automatically generated tests driver The main unit of the test driver is contained in the file mika lt file name no extension gt lt subprogram name gt driver adb lt can be compiled using gnatmake to generate the test driver executable Refer to the GUI front end section for further details especially the requirements for using this feature in terms of test points 8 Examples The example directory in the installation folder contains a number of small basic examples to ex
17. rser M C XProgram Files Midoan Software Engineering Solutions Mika bin o F vv70 code F GNAT 2009 bin gnat95 w C tmp e Ptas alarm which generates the files alarm poinC tmp alarm_ mika and alarm subprograms in C tmp ii The mika ada generator the Test Data Generator The test data generator mika ada generator generates tests according to the given criteria for a specific subprogram in a previously parsed file lt does not check if the file under test needs re parsing The basic syntax is mika ada generator M lt full path of install _dir gt S lt subprogram_name gt Tbranch Tdecision Tmcdc Cignored Cnot_ignored optional_switches lt file_name_no extension gt the M lt full path _ of install dir gt switch is compulsory and must provide the full path to the bin directory of Mika file e g M C XProgram Files Midoan Software Engineering Solutions Mika bin the S lt subprogram name gt switch is compulsory and must provide the name of the subprogram for which test data generation is desired e g SCalcEngineCoolingAir the cases are not significant To handle overloaded subprograms the optional switch 1 must also be provided Operator subprograms may be passed as S lt the operator gt e g S or S and or as Sstring lt _ascii_code gt e g Sstring 42or Sstring 97 110 100 respectively as mentioned in the GUI front
18. t the predicted outputs that will be generated by the code under test given the input tests generated you can validate the behaviour of your code without having to actually execute it if you wish Page 3 of 13 Mika provides with very little additional overheads an automatically generated test driver that uses the tests automatically generated This test driver can be compiled and run automatically Automatic testing for Ada has arrived 2 3 Requirements Microsoft Windows XP Mika is untested for other Operating Systems with NET v2 0 or above installed A working installation of the GNAT Ada compiler e g GNAT Pro GNAT GPL MinGW s GNAT Installation Download Setup _Mika msi from http www midoan com download current Setup_Mika msi and run it Follow the installation process and first use configuration dialogs 4 Ada Subset General remarks that should be kept in mind when considering the subset handled by Mika 1 There are no theoretical limits on the type of constructs that Mika could handle future versions of Mika will tackle larger Ada subset see the release notes for past subset enlargement Typically if your code contains a construct that Mika does not handle a warning will be issued and the construct will be ignored the test data generation process will continue The tests generated may still achieve the level of coverage predicted the predicted behaviour may still be correct these sho
19. uld always be confirmed using a third party code coverage tool possibly using Mika s automatically generated test driver as input The Ada subset that Mika is able to tackle is described below 8 9 The subset is based on Ada 2005 Ada tasks are not handled Access types are not handled Exceptions are not handled Tagged types are not handled Generic Packages are not handled Address clauses are not handled Machine code insertions are not handled Discriminated record types are not handled 10 Records with variant parts are not handled 11 Bitwise operators are not handled 12 Use of predefined libraries subprogram is limited Handled features of note 1 Mika does not put any restrictions on scope visibility renaming and overloading rules for the code under test 2 Unconstrained arrays are handled 3 Recursion is handled 4 Default expressions of record components subprogram parameters are handled 5 The Ada subset handled by Mika fully subsumes the SPARK Ada subset but does require any annotations 5 Limitations Because of its nature Mika may not be able to handle code of arbitrary complexity within a reasonable time with the memory at its disposal i Complexity Limitations The complexity of the control flow graph of the code under test the size of the data structures handled and the deepness of the coverage desired have a direct impact on Mika s ability to generate tests within a
Download Pdf Manuals
Related Search