(preliminary version (pdf 172k))
Contents
1. 2 Offerman A Goor A An experimental user level implementation of tcp Tech nical Report 1 68340 44 1997 07 Delft University of Technology 1997 3 Aho A V Dahbura A T Lee D Uyar M U An optimization technique for protocol conformance test generation based on UIO sequences and rural chinese postman tours IEEE Transactions on Communications 39 11 1991 1604 1615 4 Doong R K Frankl P G The ASTOOT approach to testing object oriented programs ACM Transactions on Software Engineering and Methodology 3 2 1994 101 130 5 Cheon Y Leavens G T A Simple and Practical Approach to Unit Testing The JML and JUnit Way In ecoop 2002 6 Beck K Gamma E Junit test infected Programmers love writing tests Tech nical report Java Report 1998 7 Massol V Husted T JUnit In Action Manning 2003 8 ITU T Z 140 The Tree and Tabular Combined Notation Version 3 TTCN 3 Core Language 2001 9 Pickin S Jard C Le Traon Y J z quel J Le Guennec A System test synthesis from uml models of distributed software In FORTE 2002 IFIP Int Conf on Formal description techniques Houston Texas 2002 10 Ghriga M Frankl P G Adaptive testing of non deterministic communication protocols In Protocol Test Systems 1993 347 362 11 12 13 J C Fernandez C Jard T J ron G Viho Using on the fly verification tech niques for the generation of test suites In Rajeev Alur Thoma2. 6 Q P TL U P c t d Ja P Xe x R P o t a P if 3d E P e x Rt d fem Ydi Jt e di Ad o AP 6 false otherwise TL P 8 If one possible future finishes its execution the trace conforms to the testing scenario Otherwise all the executions are waiting at least one sending autho rization If the intersection of all of these requests is not empty we authorize their emission In any other cases we cannot find a consensus Thus the test is declared false Denote that the same emission for different executions must be done in the same time because there can be only one real emission We emit therefore this message at the maximum time given by all executions With these semantics we do not assure that the emission will immediately take place when the request is made We use a best effort policy Nevertheless these semantics can be easily modified in order to emit the message immediately but it risks compelling too much the executions of the scenario Finally we just have to compute T L P 0 0 d to find if a trace satisfies a program P 2 4 An example unanimity vote We point out some of the advantages of these operators through a small example a system of unanimity vote Here the SUT is a java class that can be called to register to the electoral list an object implementing a particular interface When a question is asked to the SUT it transmits the question to so
3. stroyed getMessageIn sut message t it waits at most the time t for a particular message If any message arrives or the first message was not message the execution is destroyed getMessage suti message it makes the same thing than the previous func tion but without any timeout It is very easy to write thess fonctions For example the code of the function listenSpeech is the following function listenSpeech suti t1 t2 lt ti assert getMessage sut BEGINFDetect SpeechENDINF lt t2 assert getMessage sut BEGINFEnd SpeechENDINF be Listing 1 2 User defined function For the first test case our aim is to connect to the call API call the directory pronounce a name and verify that people picks up OABNDOKWNEH NNNNNRPREP RP BRE ee Re FPwWNnNrFOOANDOTBWNHNrF OO sut connection 1l at7290 4442 Galli sut 23 sendCommand sut RecordAll communication pcm listenSpeech sut 30 sec 30 sec sendCommand sut Talk testername listenSpeech sut 50 sec 30 sec amp amp getCommandeTimer sut Send talk done 20 sec wait 5000 sendCommand sut Talk yes listenSpeech sut 50 sec 30 sec amp amp getMessageIn sut Send talk done 15 sec sendCommand sut HangUp getMessage sut HangUp detected amp amp listenSpeech sut 50 sec 30 sec E getMessage sut HangUp detected getMessageIn sut RecordAll done 15 sec
4. amp amp getMessageIn sut HangUp done 15 sec Listing 1 3 LaTe scenario for a simple test The fact that we use an analogic line to connect to the service adds random ness in the receipt order of messages For example we do not know in advance if the message Send talk done will be received before the beginning of the speech So we have to set that all of Send talk done messages can be interleaved with the listenSpeech function A lot of executions will be evaluated but only one will fit the real events Moreover when the tester executes manually the test he can observe that sometimes the call API detects two different speeches and other times only one Thus in the scenario lines 15 to 21 we specify that these two cases can occur by using the operator We have either an HangUp detected message or a HangUp detected message interleaved with a speech detection At the end of the scenario we also specify that a HangUp done is interleaved with a RecordAll done message 5 Experimental results and discussion 5 1 Results and pros The use of LaTe in this case allowed the tester to semi automatically test the voiceXML service It was not fully automatic because the test needs an operator which verifies that his phone rings and picks up for some test cases But compared to the manual testing this solution reduces the interactions between the tester and the SUT If we compare this solution to one based on TTCN we can observe
5. amp Q o t a Po 07 max t t a ap U aq Jj gt i hp hq Projection At t R Jap aq E B Le x R Jo of h a of whg a of A Po 07 t p p P hpl of 04 t ap A Po 07 t dd Q halo 07 ot t ay A Aap dg A Gp CaAag Ca U S t t P amp amp S t t Q 07 t p dp U dq Ij gt Fhp hq Projection It t Rt Jap aq P X x RH Jo o hy a8 f halo o A P 0 t d dp P hp o of of t ap A Q 07 t o dq Q halo 07 0 t ag A ap N aq OA ap U aq a A dp U dq Q 7 We have just seen that some executions may enter in a waiting state In fact an emission is something that cannot be cancelled As a result we have to check that all executions want to send the same message Indeed in order to choose what and when an emission can be made we must have a total view of all the executions It is why the use of a top level is unavoidable It is the only element that can see the global state of the tester This component gets back the emission requests computes them in order to find if the executions are determinable After that it gives the authorization to send messages Let TL be semantics of the top level TL PP x X x Rt x Rt x P L x R x P X x Rt Bool TL is defined as follow true if Jot X x R Jt R Po 07 t
6. functional testing languages on complex SUT This paper focuses on black box testing i e the fact of testing a system where inputs and outputs of functions are known but internal code structure is irrelevant It is a form of testing which cannot target specific software compo nents or portions of the code So in the rest of this article we will use the same definitions than TTCN 8 which is the reference in this domain This language in its version 3 tries to be as generalist as possible and the most independent of the SUT SUT are more and more complex and sometimes non deterministic so we need a testing language that has to be as powerful and expressive as a program ming language This is exemplified by the evolution of TTCN Another instance is Tela an UML 1 4 based testing language 9 which gives lots of control struc tures like loops branches guards interleaving and so on However Tela is more a test description language than a test execution language New operators which are resource greedy can be proposed if they have good qualities because the time of test execution is not something important in our context For decreasing the cost of learning a new testing language they must be as easy and generalist as possible in order to test several different SUT with the same testing language e g web services java SUT etc This paper is organized as follows In the next section we present LaTe a testing language which imp
7. modify the speech recognition system so that it gives a set of possible sentences that may have been pronounced With this modification the speech recognition system will someway be non deterministic several possible verdicts will be returned Associated with our non deterministic operators it can easily find after few steps which sentence was pronounced thanks to the following sentences Thus with only few modifications of the scenarios this system will explore more deeply the real behavior of the SUT and will be more capable of detecting mistakes Another possible improvement of this system is to generate directly the LaTe scenarios from the specification One of our perspective is to modify TGV 11 for this aim TGV allows the generation of an abstract test case from a specifica tion and a test purpose The generation is done on the fly on the synchronous product of the specification with the test purpose It is based on Tarjan s al gorithm During the depth first search DFS TGV performs abstraction and determinization of this product The DFS stops when an accepting state of test purpose is reached During the backtracking TGV synthesizes the transitions of the test case Currently TGV generates test cases in both BCG 12 and Aut 13 formats so if it can be modified to directly generate test scenarios in LaTe format we will be able to reduce the work of writing these scenarios 6 Conclusion Some SUT are so complex and n
8. thing quite difficult for a test writer because he has to infer the possible state of the SUT So he has to add lots of if then elsif and if it cannot distinguish two cases he may have to indicate an inconclusive verdict Unfortunately some SUT are intrinsically non deterministic there are not fully observable so we cannot know their internal state just from their outputs To succeed in testing non deterministic SUT we have hadded two non deterministic operators in the language These operators are the non deterministic interleaving and the non deterministic choice They are presented in details in this section Note that a similar paradigm has already been used in the procol testing domain in 10 In a nutshell the solution of the non deterministic SUT problem is to have different executions at the same time for the same scenario When a divergence point is found in an execution several executions are created in order to cover all the possibilities When a contradiction is detected in an execution then this execution is stopped and destroyed The test passes when there is at least one execution that arrives at the end of the scenario otherwise it fails The first non deterministic operator we define is noted It represents a choice in a scenario For instance a 3 a 5 means that we have two exe cutions one where a equals 3 and another where a equals 5 All statements afterwards will be executed twice once for each execu
9. LaTe a Non Fully Deterministic Testing Language Emmanuel Donin de Rosi re Claude Jard and Benoit Parreaux 1 France T l com R amp D 2 avenue Pierre Marzin 22307 LANNION FRANCE emmanuel doninderosiere francetelecom com benoit parreaux francetelecom com 2 ENS Cachan Campus de Kerlann 35 170 BRUZ FRANCE claude jard bretagne ens cachan fr Abstract This paper presents a case study which is the test of a voice based service To develop this application we propose new functionalities for testing languages and a new language called LaTe that implements them With LaTe one testing scenario can describe several different executions and the interpreter tries to find the execution that best fits with the real behavior of the System Under Testing SUT We propose an operational semantics of these non deterministic opera tors Experimental results of the test of the voice based service are also included 1 Introduction The world of testing languages remains complex and dense there are often more than one language by application domain e g hardware testing 1 2 protocol testing 3 component testing 4 5 Several systems take programming lan guages in order to use them for testing purpose 6 7 One objective of this paper is to experiment with a paradigm that is non deterministic testing De spite a more complex interpretation we will prove that this can increase the quality of black box testing languages also called
10. adoi 0 ft At gt t e o t a lt if Vl c Rt Jt eRt afex 4 et canil fE At lt t e 0 t a et otherwise In order to execute P Q P is executed first with the current environment Then for each possible future where all the statements were executed Q is exe cuted with the new environment For each future where a sending authorization is awaiting the remaining program is rebuilt P Q o t al IQ 07 7 a Po t t a o P o t al ULHO 07 t a d P 07 t a d P o t a AP A Po 5 The possible futures of P Q are the union of possible futures of P with those of Q PIlQ 0 t a P o t a U Q 0 tq 6 The interleaving operator amp amp is the most complex operator in these se mantics In order to execute P amp amp Q P and Q are executed separately To do this two disjoint sets ap and aq are extracted from a Each set corresponds to the part of a used by each process gf g is also divided into two parts thanks to two projections hp and hg If the two processes are fully executed the execution is well finished and the new current time is the maximum of the two final times If at least one of them is waiting for a sending authorization the state of the program is rebuilt at the moment of the send of the message Therefore waiting times S t t and S t t have to be added in order to take the passing time into account P amp
11. conversation will be saved in a PCM format BECMDHangUp ENDCMD sent when the tester wants to hang up BECMDEnd ENDCMD sent when the tester wants to stop the communication be tween it and the calling computer For each of this message the call API sends a corresponding done message when the operation executed without any problem or a not started when a problem occurs Moreover the call API can emit particular messages BEGINFDetect Speech ENDINF it is sent when the call API discovers that there is someone who speak during the conversation BEGINFEnd Speech ENDINF it is sent when the call API discovers that the speech finishes BEGINFHangUp ENDINF sent when the call API discovers that the line has been hanged up At first we tried to allow the API to use vocal synthesis in order to send every possible message However we discovered that the speech recognition system of the voiceXML service has lots of difficulties to recognize generated voice These problems increase the number of cases we have to manage in our scenarios so we finally preferred to record the sentences we will play during the tests for this reason We also imagine using a speech recognition system for our call API Indeed if we can know what was pronounced by the service we can deduct in which oRWNH state is the SUT Nevertheless as for the previous remark speech recognition has difficulties to recognized generated voices So lots of recognized sentences w
12. ere wrong because we were comparing character to character these sentences with the sentences of the specification We may modify the speech recognition system in order to give for each sentence pronounced by the voiceXML service the sentence of the specification that may have been pronounced With respect to our aim we develop a socket SUT adaptor factory Thus in LaTe you just have to specify the host and the port of the server and LaTe automatically creates the stub adaptor and connects itself to the server The code of this factory is quite simple but we will not show it here because we are not specially interested by this information in this paper Just with these information we allow LaTe to easily test the vocal based directory So in the next section we will see in details some scenarios and further the advantages of the non deterministic operators on this particular test case 4 2 Some test cases For all of the following test cases we defined several LaTe functions to simplify the writing of scenarios These functions are connection host port this function initializes the communication between the tester and the call API call sut num for a giving SUT it send a message for calling the phone number num sendCommand sut command it sends the corresponding command to the SUT listenSpeech sut t1 t2 it verifies that a sentence is prononced before the time t1 and during at most the time t2 Otherwise the execution is de
13. ill be studied in details the voice based service that we will just study the vocal based telephone directory 3 Description of the case study testing a voice based service We decided to validate our approach on the test execution of voice based services and particularly a vocal activated telephone directory In this service the user gives a name and the service seeks this name and then proposes to put the user in relation with the found number In the case of homonyms the service proposes several solutions and requests the user to choose the solution which is appropriate to him The new voice based services use intensively speech synthesis and recogni tion These functionalities simplify the access to the voice based services but complicate their validation Indeed they produce lots of non determinism if we try to automatically test it For example the volume and the speed of the voice during two different conversations can change If we need to recognize auto matically the sentences pronounced by the voice based services using a speech recognition tool we should make the verdict even more random Furthermore the use of speech recognition for the tester also increases the non determinism of all the system Many factors can affect the result of the speech recognition such as the quality of the transmitted messages and the qual ity of the line and so on It is possible for a same message to be correctly rec ognized the first time but no
14. in any order without changing anything This can be true only if a statement of a particular branch cannot influence other branches In order to prevent this we implement a locking variable system if a variable is modified in a particular branch it cannot be read or written in other branches With all these restrictions we can verify the initial hypothesis As we said before we need a component that can view all the executions in order to decide which messages can be sent and when This component will be called top level in this article Each time an execution wants to send a message it sends a request to the top level If all the executions want to send the same message the top level emits it and wakes up all the executions If all the execu tions want to emit different messages then the top level may choose randomly an execution sends the corresponding message and destroys the other messages or may stop the evaluation of the scenario and returns an error depending of the configuration of the LaTe interpreter 2 3 Operational semantics In the following equations P S denotes the set of all subsets of S e the con catenation operator and w the shuffle product Let be the following semantic operator P x 2 x Rt x Rt x P L x Rt gt i P P x Xr x Rt x Rt x B L x R x P e x R P corresponds to the set of all programs formed by waiting sending and receiving statements and interleaving alternative and sequence operat
15. lements non deterministic operators We also give the semantics of these operators Then we give some information about the case study testing a voice based service and the actual difficulties Section 4 is dedicated to the test architecture of the experiments and to the examples of test cases in order to show the pros and cons of the constructions proposed here Finally we discuss the results and conclude 2 Presentation of LaTe One of the aims of this study is to show that non deterministic operators can be very useful in testing languages In order to evaluate these new operators we have created a new language but they could be added in a more complete language like TTCN In this section the main characteristics of LaTe will be presented Then a description of the non deterministic operators will be given Finally we will see the power of these operators and LaTe through a small example of unanimity vote 2 1 Main requirements In Late we try to select some important features Genericity the language should be able to test different SUT It is unfortunate to have to learn a different testing language for each system you want to test So it is better if an unique language can test every SUT may be through an adaptation layer As in TTCN the use of SUT adapters can be useful Nevertheless unlike TTCN it is interesting to write only one SUT adapter to test different SUT of the same type e g one SUT adapter for testing all the SUT
16. me electors These calls can be executed concurrently and if one of the electors replies false the returned value of the SUT will be false and all the electors may not be asked to vote it is a unanimity vote This example is quite difficult to test using classical systems because it has to use stubs for electors and has to control each stub some of the interactions are concurrent We have to verify if the test scenario agrees with all the possibilities of interleaving the SUT interactions it contains lots of non determinism if one of the electors replies false the SUT may continue to call all of the electors but it also can stop and directly replies to the tester The test scenario must describe all of these cases ANowkWNH With LaTe it is quite simple to write a testing case for this SUT For example for 2 electors the scenario may be the following mysut createStub Java SUT elector1 createStub Java ElectorInterface callSut mysut register elector1 We create a stub and register it to the SUT elector2 createStub Java ElectorInterface callSut mysut register elector2 We create a stub and register it to the SUT sutcall callSUT mysut ask Is 42 prime it lt 10 sec stubcall1 getCalled elector1 replySUT stubcalli false The SUT asks the question to the elector 1 and we reply false amp amp id lt 10 sec stubcall2 getCalled elector2 replySUT s
17. must separate the behavior of the stub and the tester component the tester has to send a message to the stub in order to inform it that the call to the SUT have been made and it will receive a call from it With our scenario it is more compact and we can easily show the event succession Powerful API SUT become more and more complex and non deterministic A large collection of APIs is then needed to check that the value returned by the SUT is in accordance with the specification The easiest way to have a large collection of API is to allow testers to use API from a programming language In our language we can easily use the Java APIs Dynamic language just like above some SUT can create dynamically PCO Point of Control and Observation so we need to discover them during the execution of the scenario and dynamically connect to these PCO Something like a scripting language sometimes the tester needs to test in real time the SUT For example when the tester debugs a test scenario he may need to have a console to execute his own commands So an interpreted language will therefore be more useful than a compiled one Moreover it will be easier to have a dynamically typed language in order to write quickly the scenario and avoid the check and cast of values each line 2 2 Non deterministic operators As mentioned above SUT become more complex and this complexity leads to more non determinism However the non determinism in a SUT is some
18. nnot emit A because there is an execution that does not want to send something We are in a livelock In order to resolve this conflict the scenario must contain information about timeouts in all the receive instructions e g if the b lasts more than 5 seconds this execution will be destroyed and there will be only one possible execution the first one The evaluation of the scenario will continue In other case if the SUT sends the two messages a and then b without waiting for the replies A or B there will be another problem Three executions will be then possible They are displayed with dash in Figure 1 Two of these executions want to send A and the last intends to emit B This is because of the exact symmetry of the test scenario there is no difference between the two stubs In this case we decide that LaTe may choose randomly an execution and continues it Then LaTe prints a warning message in order to warn the tester that the scenario contains conflicts LaTe may also stop the execution according to its configuration Through this example we can see some of the advantages of the non deter minism operators of LaTe they permit to easily describe the parallelism and the non determinism of the SUT Although this example is an extreme case of non determinism associated with concurrency we can test quite easily this behavior Nevertheless the tester has to think of all possible executions In the next sections a more complex example w
19. on deterministic that usual testing systems and languages have difficulties to evaluate the state of the SUT Thus we have de fined two particular operators for testing languages the non deterministic choice and the non deterministic interleaving These operators allow the tester to main tain several different executions at the same time Each execution is independent of the others and is destroyed when a contradiction is found Nevertheless com munications between these executions and the SUT must be managed because of the communication cannot be undone and modify the environment of all ex ecutions In order to show that these non deterministic constructions may be useful we have implemented them in a new language LaTe and we have applied them on a particular case study testing a vocal based telephone directory They were par ticularly useful on this case because the SUT contains lots of non deterministic behavior like interleaved events optional messages etc Thus these constructions allowed to increase the automation of this task and also allowed the scenarios to test deeper behavior than usual test scenarios Finally we have proposed several enhancements for this particular case study like adding a speech recognition system in order to increase the power of the system and test deeper voice based services References 1 Thomas D E Moorby P R The Verilog Hardware Description Language 3rd edn Kluwer Academic Publishers 1996
20. ors It also contains the null program Po Diy is the set of messages that can be sent by the SUT and Xe is the set of all messages sent by the tester This operator expresses the set of all possible futures from a program P a trace o containing the received messages and their arriving times the current time and a list of sendable messages Each future is made up by the remaining program to execute which can be null the current time the list of sendable messages and the set of messages to send If the remaining program is null then all the statements were executed else the execution is waiting for the authoriza tion to send a message to the SUT The null program can be executed but does not modify the context Po 0 t a Po 0 t a 2 let e lt be the statement meaning that the next message received must be e and that it must arrive before w seconds If it is true o 0 e7 and t w gt 7 then e is deleted from the trace and the clock is moved forward by 7 Po o 1 t 7 a ifo 0 e7At w gt rT 0 otherwise e lt 0 t a 3 Let F be the statement that means E must be sent to the SUT First if belongs to the set of sendable messages it is sent If not the execution is stopped and E associated with the current time is added to the list of messages we want to send We also verify that any more incoming message does not arrive Po o t a e 6 if t t CR afe S e E
21. re Thus we develop a special test architecture in order to allow the tester to connect to the voiceXML service This architecture is represented in Figure 2 The tester represents the machine where LaTe and the scenarios are executed It communicates through a socket with the call API This computer is linked to a call card which can makes calls and conversation on an analogic line Therefore this computer can call the voice based service As we just said there is a dialog between the tester and the call API through a socket A special protocol has been defined to allow the tester to test the service and obtain information about the conversation This protocol uses all of these messages Cc vray Ca CI A lk A VoiceXML Service Socket Analogic line O O Tester Call API Fig 2 Test Architecture BECMDCall num ENDCMD this message asks the call API to call the number num BECMDDTMF num ENDCMD it asks the call API to simulate the pushing of a par ticular touch defined by num in the telephone keyboard This is done by emiting a special sound called DTMF during the conversation BECMDWaitDTMF num ENDCMD this message asks the API to wait a particular DTMF sent by the directory BECMDTalk file ENDCMD it asks to play a particular file during the conversa tion This file must be in a PCM format at a good frequency BECMDRecordAll file ENDCMD it asks the API to record all of the conversation in the file file This
22. s A Henzinger eds Proceedings of the Eighth International Conference on Computer Aided Ver ification CAV Volume 1102 New Brunswick NJ USA Springer Verlag 1996 348 359 Tock L P The bcg postscript format Technical report INRIA Rh ne Alpes 1995 Fernandez J C Aldebaran user s manual Technical report Laboratoire de G nie Informatique Institut IMAG 1989
23. t the following times Thus the presence of speech recognition and synthesis causes a significant number of inconclusive verdicts during automatic tests One current solution for this problem is to replace all speech signals emitted and expedted by the platform by DTMF Dual Tone Multi Frequency signals These signals are the sounds produced when you dial a number with your phone This solution is not completely satisfactory because it requires the modification of the voice based service The verdict of the tests can also be deteriorated by these modifications The other solution is to carry out the tests taking the risk of obtaining a significant number of inconclusives verdicts No other solution are possible with common testing languages The language that we propose in this article enables us to bring a new solution to this problem by using non deterministic operators in the test scenario So this vocal based phone directory although very simple is enough to clarify the interests of our language 4 Methodology of the experiment In this section we will see in details how the vocal based telephone directory was tested using LaTe First we will see the test architecture of this experiment then we study the communication protocol between the tester and the calling platform Finally some test scenarios will be presented 4 1 Test architecture In section 3 we discussed in details how works the vocal based telephone direc tory testing he
24. that TTCN contains an interleave statement that specifies that different branches must be executed concurrently TTCN allows user to write its own functions in ternally or externally nevertheless they can be used in an interleave branche In this particular case LaTe is more powerfull than TTCN because we don t have to inline the getMessage and listenSpeech functions Moreover TTCN does not allow to specify in a test scenario that something is optional The only statement that can be used for that is the alternative one but the user have to give a guard for each branches of the alternative statement which is very difficult in our example Another advantage of this testing architecture is that we have to our disposal all the traces of the conversations between the tester and the voiceXML service So when we discover an error we can easily locate it thanks to these traces 5 2 Discussion As we said previously this method for testing the vocal based phone directory is not perfect the tester does not know exactly what is pronounced during the conversation and he has to pre record all of the sentences before using it Thus one possible evolution of this technique is to use speech recognition We saw in section 4 1 that a normal system may not work for our example The recognition is not something perfect and may make mistakes So some verdicts may be fail with any difference between the specification and the SUT One possible solution is to
25. tion It can be done because the statements in one execution are independent of the ones of another execution However communications between the tester and the SUT do modify the state of the SUT so precautions must be taken before executing these instructions In LaTe when an execution wants to send a message it always requests it to a component that can view all the current executions The other non deterministic operator is noted amp amp It represents a non deter ministic interleaving It executes all of the possible interleavings of two given branches For example A B amp amp C D is equivalent to A B C D A C B D A C D B I11 C D A B C A B D 1 1 C A D B This operator helps to easily describe two independent behaviors For ex ample when stubs are used in a scenario it allows to specify that the behav ior of one is independent of the other Nevertheless the longer the branches are the more different executions are evaluated Thus in order to decrease this number we decide that only communication with the SUT statements will cause the evaluation of new execution For instance with the following scenario a assert a 5 amp amp b assert b 6 only two executions will be evaluated a assert a 5 b assert b 6 and b assert b 6 a assert a 5 instead of the 6 possible interleavings We can do this because we suppose the instructions that do not communicate with the SUT can be executed
26. tubcall12 true The SUT may ask the question to the elector 2 and we reply true But it is optional F assert getReply callSut false The answer is false Listing 1 1 LaTe scenario for the unanimity vote As mentioned above LaTe evaluates all the possible interleavings of the sce nario In other words whatever the order of stub calls by the SUT and whatever the number of called stubs LaTe is capable of verifying that the execution fits the specification described in the scenario Time specification can also be easily added by using the operator Nevertheless LaTe may have difficulties to evaluate particular scenarios For instance the pseudo code a A amp amp b B is problematic if the SUT sends the two messages a and b without waiting for the replies A or B because this behavior is extremely non deterministic All the executions are presented in Figure 1 In this example if the SUT sends a and waits for A before sending b there is also a problem All the branches beginning by b are not executable because the first message received by the tester is a and the assertion b which signifies that the next message receive by the tester is b will be broken Thus in the next step two executions are possible b and A The second execution with b is in a waiting state because the SUT does not send an additional message and Fig 1 All the possible interleavings the first execution ca
27. written in JAVA one SUT adapter for Web services Actually three SUT adaptor factories have been written the Java one the C one and the socket one and They contain respectively 250 400 and 100 lines of code It was very easy to write them because of the use of reflection in Java and the Java Native Interface that allows use C functions in Java code Using stubs It is sometimes useful to develop stubs in order to test some SUT But in most languages the user needs to write a specific stub for each test case In addition the interactions between the SUT and the tester and those between the stub and the SUT are often described separately Thus it is necessary to add synchronization points in the scenario in order to express precedence between an action of the tester and one of the stub With LaTe like for SUT adapters you need to write only one stub constructor to create stubs for a specific type of SUT and all the interactions between stubs and the SUT are directly written in the global scenario For example the following code specifies the creation of a Java stub that implements the interface MyInterface mystub createStub Java MyInterface You create a stub callSut mysut register mystub You send it to the previously created sut stubcall getCalled mystub You verify that the stub is called by the sut This example shows that you can easily describe the general behavior of the system In a system where you
Download Pdf Manuals
Related Search