More on Cucumber: Steps, Scenarios, and Troubleshooting
Contents
1. Ken likes to play soccer and nothing else Kenneth M Anderson 2012 25 Regular Expressions X e Parens are also used to specify capture groups e That is they capture what was matched inside of them and make the captured pattern available for later processing e There are 1 9 0 9 cows in the field e The above expression matches sentences like e There are 10 cows in the field or There are 19920 cows in the field e Also unfortunately There are 1 cows in the field e AND makes the actual number available e In a step definition a captured pattern is passed as an argument to the step definition s method body Kenneth M Anderson 2012 26 Example from Lecture 9 e Given Athe input 4 do arg1 e input arg e end e We now should understand the regular expression better e the input appears at the start of the step followed by a quotation mark e match any character that is not a quotation mark zero or more times e the parens around the above pattern captures the result as arg e the step must end with a quotation mark Kenneth M Anderson 2012 27 More about Steps l e Any step can be augmented by a data table e These are not the same as the table that appeared when using a Scenario Outline within a feature e Instead it is a table that appears immediately after a step like this Then my shopping List should contain Onio2. handle_output rb each invoke rb 2 directories 5 files Kenneth M Anderson 2012 11 Matching Steps V e This also means that you have to be careful with how you write your steps e Scenario 1 P e Scenario 2 e Given have 100 in my Account e Given a starting balance of 20 in e When I request 20 my Account e Then 20 should be dispensed e When deposit 80 e And my balance is 80 e Then have 100 in my Account What s the problem Kenneth M Anderson 2012 12 Matching Steps V e This also means that you have to be careful with how you write your steps e Scenario 1 7 e Scenario 2 e Given I have 100 in my Account e Given a starting balance of 20 in e When I request 20 my Account e Then 20 should be dispensed e When deposit 80 e And my balance is 80 e Then I have 100 in my Account These two statements are treated as equivalent by Cucumber but in one case it is being used to initialize a scenario in the second case it is being used to assert that something is true of the scenario Kenneth M Anderson 2012 13 Matching Steps VI e When you have two equivalent steps e Given have 100 in my Account First Step used to initialize e Then I have 100 in my Account Last Step used to assert e they will both cause the same step definition to be invoked e Given I have 100 in my Account do e lt code goes here gt e end e The problem is that lt code goes here
3. you can classify both features and scenarios with tags Kenneth M Anderson 2012 33 Tags l e A tag is a word prefixed by the character that can appear on the line before either the keyword Feature or the keyword Scenario e A tag on Feature will be inherited by all of that feature s scenarios e Example e employee e Scenario Add an employee e Given Ken is a Person e And Ken is accepted for a job at our company e Then Ken is added as an employee e This scenario is now tagged with the keyword employee Kenneth M Anderson 2012 34 Tags Il e You can have multiple tags separated by spaces e Example e slow widgets nightly e Scenario Generate overnight report e Given am logged in e And there is a report Total widget sales history e This scenario has three tags slow widgets and nightly Kenneth M Anderson 2012 35 Tags III e Now that you have tagged scenarios they become useful because you can ask cucumber to run just the scenarios with a particular tag e cucumber tags nightly slow e Cucumber will now run only those features and scenarios that have been tagged with the nightly tag e This enables you to raise the quality of your develooment process by configuring a continuous build system to invoke this command at night and log the output for review the next day Kenneth M Anderson 2012 36 Troubleshooting Cucumber e The authors of the cucumber book identify e
4. four types of problems that can be encountered e when trying to incorporate behavior driven design into a life cycle e Those problems are e Flickering Scenarios Tests are unstable some randomly fail e Brittle features Changes to the system cause existing features to break e Slow features The test suite takes too long to run e Bored stakeholders Our customer is no longer creating reading features Kenneth M Anderson 2012 37 Flickering Scenarios e A flickering scenario is one that every now and then fails randomly e The unpredictable nature of the failure reduces team confidence e This uncertain situation in turn reduces the desire to run the test suite e The biggest problem with this situation is that e you cannot fix the situation if you cannot get the bug to be reproducible e Contributing Factors e Shared Environments e Leaky Scenarios e Race Conditions and Sleepy Steps Kenneth M Anderson 2012 38 Shared Environments e Shared Environments e Multiple people use the same machine to test in parallel e The tests of the two users have the potential of clobbering each other e Creating editing the same database at the same time writing to a shared XML file etc Boom e Solution e Use techniques that isolate one instance of a test from another instance of that same test e For instance have the test create a tmp directory unique to it where it stores all of its data e Multiple instances of that same tes
5. to the customer e developers must work with testers e testers will be looking for corner cases and good coverage e developers can use their expertise to ensure that the test cases are properly decomposed and help with refactoring duplicate behavior Kenneth M Anderson 2012 Perspective Ill e All three stakeholder roles are needed to balance each other e customers ensure that tests are in scope and important e developers ensure that tests are well maintained e also help to ensure that all the information needed to run a test is present e testers ensure that the test set is comprehensive e and that we are not ignoring certain tests because they are hard e The first principle is that you must not fool yourself and you are the easiest person to fool Richard Feynman Kenneth M Anderson 2012 Review and two new tidbits calculator calc rb lt __ System Under Test features Features Directory adding feature lt Feature step_definitions lt _ Step Definitions Dir L calculator_steps rb_ Step Definition File 2 directories 3 files e Cucumber conventions lead to the above folder file hierachy e There can be multiple directories under features to help organization e There can be multiple files in each step_definitions directory to help organize the code that implements the step definitions Kenneth M Anderson 2012 7 Matching Steps e In lecture 9 we e
6. More on Cucumber oteps Scenarios and Troubleshooting CSCI 5828 Foundations of Software Engineering Lecture 13 02 28 2012 Kenneth M Anderson 2012 Goals e Review material from chapters 4 6 of our testing textbook e Learn more about Cucumber and how it supports behavior driven design e More about Steps and Step Definitions e More about scenarios e Examining typical problems encountered with Cucumber e Review additional examples throughout Kenneth M Anderson 2012 Perspective e Cucumber is aimed at integration and acceptance testing e t is a testing and communication tool for expressing e end to end tests that cover the major capabilities of your system e view to controller to model and back e Ul to database and back e tests that touch all of the subsystems of your system these subsystems might exist on multiple nodes e As such you will still write unit tests for your system using some other testing framework and run them alongside cucumber based tests e Your customer will only be involved with the latter Kenneth M Anderson 2012 Perspective Il e To make this work cucumber requires discipline to ensure that the right people work together to create the integration tests e customers must work with developers e to prevent the developers from writing tests that are too low level e to ensure that tests are written using customer terminology e to ensure that what is being tested is important
7. Tester Apartheid e Testers are often regarded as second class citizens on a software team e They may not have as much technical or software engineering skills as developers but they are e capable and good at writing automation scripts e good at coming up with corner cases e good at coming up with comprehensive test suites e However if they do not work with developers their test code can degrade if it is not properly maintained e Solution Have testers and developers work together and encourage this as part of company culture developers can refactor the test suite when needed and learn from the testers about how to best test their code Kenneth M Anderson 2012 45 Slow features e After creating a lot of tests it takes a long time for the entire test suite to run e You accumulate tests because you want to know when a change has broken previously passing tests You can back out the change and or figure out how to fix the regression e When test suites take a long time to run e developers shy away from running them and as such they start to commit their changes without testing them e This leads quickly to a situation where a broken build is the norm e Contributing Factors e Race Conditions and Sleepy Steps See above e Lots of Scenarios e Big Ball of Mud Kenneth M Anderson 2012 46 Lots of Scenarios e Lots of scenarios will of course lead to slow test runs e t takes a certain amount of time for each s
8. This is touted initially as a way to create more abstract steps e A step that says Given the account is activated for Ken might delegate to e Given the account is created e Given the account has a balance of 50 e Given the account has an owner named Ken e Then Ken activates the account e But the book ends up strongly warning you away from this feature Kenneth M Anderson 2012 31 More on Scenario Outlines e A scenario outline can have more than one table of examples e Scenario Outline Withdraw fixed amount e Given I have lt Balance gt in my account e When I choose to withdraw the fixed amount of lt Withdrawal gt e Then I should lt Qutcome gt e And the balance of my account should be lt Remaining gt e Examples Successful withdrawal Balance Withdrawal Outcome Remaining 500 50 receive 50 cash 450 500 100 receive 100 cash 400 e Examples Attempt to withdraw too much Balance Withdrawal Outcome Remaining 100 200 see an error message 100 0 50 see an error message 0 Kenneth M Anderson 2012 32 staying Organized e When creating features and scenarios cucumber offers two mechanisms to help organize them e First as already mentioned you can have as many subdirectories under the features directory of a cucumber project as you want e features e sorting e adding_employees e calculating_payroll e Second
9. ates user Kenneth M Anderson 2012 51 Ubiquitous What e The team has failed to incorporate the language of the customer and their application domain into the system design and project culture e If you are developing a ticketing system you might have words in your system like concert performance artist venue e If you ignore those terms and use arbitrary or terms so generic that there is no obvious mapping or the terms could be mapped to anything e then your customer can become discouraged and disengaged e Instead encourage your team to develop and use a language which is shared with the customer e jt will reduce mistakes and misunderstandings improve team confidence and morale and foster strengthen the relationship with the customer Kenneth M Anderson 2012 52 Siloed Features e Cucumber is a command line tool and the features it processes are text files stored in the file system and checked into configuration management systems e As a result it is very easy for the features to hide from the customer e They might not feel like they can access the features easily e Access might require the use of unfamiliar tools git text editors e Solution e Publish the features in a way that your customer can access them e Use scripts for instance to convert them to HTML and share them with the customer via a website e Engage with the customer to ensure they are always reading writing the features and sce
10. cenario to run and that adds up e Often however this is a symptom of the system architecture e For instance a big monolithic system might require all the features scenarios to live in one place and all be tested together e Solutions e Decompose the system architecture and have features that target just individual components and then add features that target inter component interactions e Divide feature folders into hierarchies and tag features so that subsets can be easily run independently from one another Kenneth M Anderson 2012 47 Big Ball of Mud e No software design has been applied to a system at all e My friend once encountered a system which implemented shopping cart functionality for websites e it consisted of a single method that when printed covered 42 pages e These systems have low cohesion one component doing too many tasks and tight coupling too many dependencies between components e As a result its difficult to test just one thing and your scenarios will have lots of unintended duplication slowing things down e Solution e Refactor refactor refactor e Have the team focus on the architecture of the system for an iteration or two Kenneth M Anderson 2012 48 Bored Stakeholders e Stakeholders become disengaged with the process of developing the test suite that is needed to help guide development e They no longer read existing features e They no contribute to th
11. e creation of new features e They are unwilling to meet with the development team e Contributing Factors e Incidental Details e Imperative Steps e Duplication see above e Ubiquitous What e Siloed Features Kenneth M Anderson 2012 49 Incidental Details e Scenarios contain a lot of detail that are not relevant to what is being tested e The book presents an example that is testing whether an e mail is received after it has been sent e The original example had steps that declared the passwords of the users but these passwords were never used e The example was rewritten to be much shorter by abstracting away most of the incidental details and leaving clear what exactly was being tested e Solution e Always ask yourself if you are writing at the right level of abstraction e Do not let yourself be influenced by existing step step definitions Kenneth M Anderson 2012 50 Imperative Steps e Imperative steps are ones that are written in the style of do this do that e The problem is that it is very easy for the steps to be written at too low level of abstraction containing lots of unnecessary detail e Declarative steps are written at a higher level of abstraction and allow the programmer leeway in how they are carried out e Contrast this e User is not logged in He goes to home page He is redirected to login page e with e User is not authenticated He tries to view restricted content System authentic
12. gt will do the same thing each time it will not be able to customize what it does based on the two different contexts e How to fix Kenneth M Anderson 2012 14 Matching Steps VII e To fix this problem you need to rewrite the steps e Given have deposited 100 in my Account e Then the balance of my Account should be 100 e Now it will be clear that e the first is used to initialize the Account e and the latter is used to verify the account s balance Kenneth M Anderson 2012 15 The Matching Process e When invoked Cucumber reads in all of the step definitions that it can find e Each step definition causes a pattern to be registered with Cucumber e t then starts to process each feature file looking for scenarios e For each step in a scenario it checks to see that it matches one of the registered regular expressions e f a match is found the code associated with the step definition is executed and a result is recorded e f a match is not found the step and scenario is considered undefined e The next step is then processed assuming the previous step passed e Otherwise the scenario either failed due to an exception in the step or the scenario is marked pending because the step itself was marked pending Kenneth M Anderson 2012 16 Regular Expressions l e Regular expressions are a mechanism for specifying patterns that can appear inside text documents e Each expression can consist of e reg
13. h e ate four slices of pizza e ate five slices of pizza e ate nine slices of pizza e Note since we wanted to make sure that a period appeared at the end of a matched sentence we explicitly matched the period by escaping the last period in the regular expression with a backslash e Otherwise the expression would match I ate nine slices of pizza Kenneth M Anderson 2012 20 Regular Expressions V e If you need to specify that any one of a particular set of characters might appear in a particular spot in a regular expression you use a character class e A character class is specified using square brackets and then can list one or more ranges of characters assuming ASCII ordering e There are 23456789 cows in the field e Matches There are 3 cows in the field but not There are cows in the field e If characters appear in Sequence you can use a hyphen to express a range e Your id number is A Z A Z O 9 0 9 0 9 e Your id number is BZ232 Kenneth M Anderson 2012 21 Regular Expressions VI e Beware unintended inclusions of characters when using the hyphen e a Z is an empty range and A z includes P Y T and IN e Instead you need to do a zA Z ASCII Code Chart Kenneth M Anderson 2012 22 Regular Expressions VII e More on character classes e If you want to match all characters BUT the ones listed start the class wit
14. h the character e 40 9 match any character that is not a digit e If you need to match a hyphen list it first e A Za z match any letter upper case or lower case or a hyphen e If you need to match a character list it in any position but the first e A Z a z e Some character classes are predefined s whitespace d digit etc e See page 49 of the testing textbook for examples not required Kenneth M Anderson 2012 23 Regular Expressions VIII e Patterns can be tagged with repetition modifiers e the preceding pattern can appear zero or more times e the preceding pattern can appear one or more times e the preceding pattern can appear zero or one times e Alternative choices for a pattern can be separated by the pipe character e Parens and can be used to group patterns for alternation e There is are 0 9 cows in the field e Matches e There are 2 cows in the field and There is 1 cow in the field but also There is 5 cow in the field and There are 999999993421 cows in the field Kenneth M Anderson 2012 24 Regular Expressions IX e Patterns can be anchored e at the beginning of a regular expression anchors it at the beginning of a line of text e at the end of a regular expression anchors it at the end of a line of text e Ken likes to play soccer e This regular expression matches only the string
15. narios with the development team Kenneth M Anderson 2012 53 Summary e We learned more about Cucumber e Steps and the step matching process e Regular expressions and their use in steps e Scenario Outlines e Tags e We also learned about some of the problems that can be encountered when executing behavior driven design e and solutions that can be used to address those problems Kenneth M Anderson 2012 54
16. ncountered steps that looked like this e Scenario Attempt withdrawal using stolen card e Given have 100 in my account e But my card is invalid e When request 50 e Then my card should not be returned e And should be told to contact the bank e t turns out the step keywords Given When Then And But are for humans only cucumber doesn t care what you use Kenneth M Anderson 2012 Matching Steps Il e Indeed this is an equivalent scenario e Scenario Attempt withdrawal using stolen card have 100 in my account my card is invalid request 50 my card should not be returned should be told to contact the bank e The is simply used to denote a new step Kenneth M Anderson 2012 Matching Steps Ill e The reason this is equivalent is that cucumber uses just the highlighted text e Scenario Attempt withdrawal using stolen card e to match a step to its step definition e All step definitions are read in at run time and then regular expressions are used to find a match Kenneth M Anderson 2012 10 Matching Steps IV e As a result it does not matter how you organize your step definitions e Here is a version of calculator that splits its step defs across three files The behavior of Calculator alt cucumber is calc rb identical to the features previous config adding feature step_definitions Three files with hand le_input rb one step def
17. ns Potatoes Sausages Apples Relish e The step definition will then contain an argument where this table is passed as a first class object Kenneth M Anderson 2012 28 More about Steps Il e The details of what you can do with the table is provided in the cucumber documentation e lt http cukes infto cucumber api ruby latest Cucumber Ast Table html gt e The book provides a basic example using Tic Tac Toe e DEMO e It also hints at what can be accomplished e Given these Users name date of birth Michael Jackson August 29 1958 Elvis January 8 1935 John Lennon October 9 1940 e A step definition could process this table at run time and create 3 instance of the User class configured as shown and stored in a collection users Kenneth M Anderson 2012 29 More about Steps Ill e Any step can also be augmented with a doc string e Then I should receive an email containing Dear Sir You are no longer subscribed to our mailing list Sincerely SpamIsUs e The entire contents of the doc string will be passed to the step definition e Your code can then store the string or manipulate parse it using any of ruby s string manipulation capabilities e We ll see examples later this semester Kenneth M Anderson 2012 30 Nesting Steps e You can have a step definition that turns around and invokes other step definitions e This is called nested steps e
18. refactored e Contributing Factors e Fixture Data e Duplication e Leaky Scenarios see above related to dependencies between tests e Tester Apartheid Kenneth M Anderson 2012 42 Fixture Data e Fixture data refers to having a large amount of data stored somewhere in your test environment that all tests share and come to depend on e A change in that data can cause tests to fail because developers fail to realize that lots of tests depend on it e Large sets of fixture data can slow test suites down if all of the data has to be loaded for each scenario e Solution The book recommends an approach called test data builders in which all the data for a particular test is created by the test itself e t points to a ruby based framework called FactoryGirl as an example of this approach e https github com thoughtbot factory girl Kenneth M Anderson 2012 43 Duplication e Duplication refers to having multiple features that test the same thing e Duplication can e make scenarios brittle one change breaks multiple scenarios e slow your test suite down as the same functionality gets tested again e and make your customers bored can t attach significance to features e Solutions e Make use of the Background and Scenario Outline keywords e Watch out for steps with low abstraction e User clicks on next button to go to the next page vs Users navigates to Accounts page Kenneth M Anderson 2012 44
19. t can now be run in parallel Kenneth M Anderson 2012 39 Leaky Scenarios e Leaky Scenarios e One test creates an environment that another test depends on e The tests have different tags and cucumber gets invoked on just the tag of the second test Boom e Someone changes the first test not realizing that a dependency exists Boom e Solution design tests to create everything they need from scratch e Have a really complex system Use mock objects to simulate non essential parts with respect to the test of the system Kenneth M Anderson 2012 40 Race Conditions and Sleepy Steps e Race Conditions and Sleepy Steps e You have a complex system and your integration test causes two parts of the system to run in parallel e or the system to run in parallel with cucumber e The test will pass when only the right component finishes first e f the race is close you end up with a flickering scenario e Developers combat this by causing certain steps to sleep to wait for the concurrent operation to end hence sleepy steps e Solution You need to engineer synchronization points for cucumber that ensures it waits for a system component to finish its work before testing it Kenneth M Anderson 2012 41 Brittle Features e Brittle features are ones that break at the slightest change to other parts of the system e The design of the underlying system may be too tightly coupled and will need to be
20. ular characters e metacharacters e groups e anchors Kenneth M Anderson 2012 Regular Expressions II e A regular expression in ruby begins with a slash and ends with a slash e Ken e All regular expressions start and end with the metacharacter Metacharacters have special meaning in this case slash acts as a delimiter e The above regular expression consists of three regular characters K e and hn in that order e It would match these sentences Here the regular expression is unanchored and so it will match any sentence that contains the e Pete called Ken on Tuesday string Ken e Ken is a faculty member e but not this sentence e Dirk is a faculty member Kenneth M Anderson 2012 18 Regular Expressions Ill e If you want a metacharacter to act like a regular character you must escape the metacharacter using a backslash e Ken VPete e This expression would match the sentence e The meeting will be led by a faculty member Ken Pete e But not e Ken will lead the meeting e Since backslash is a metacharacter if you want to match it you need to escape it with you guessed it another backslash character e In LaTeX use cite to reference a journal or conference paper Kenneth M Anderson 2012 19 Regular Expressions IV e A period is a metacharacter that will match any character in the text file e ate slices of pizza e will matc
Download Pdf Manuals
Related Search