04 Debugging - Department of Informatics
Contents
1. Possible ways O Analysis of problem description O Static analysis of the code O Analysis of a erroneous execution run O Comparison of correct and erroneous execution runs O Building new hypotheses on the basis of previous ones e Must be compatible with previously accepted e Must not use assumptions that stem from previously rejected hypotheses Software Quality 4 Debugging 2014 Martin Glinz 32 Derive and check predictions O Techniques Static or dynamic analysis of the code e Observation of system states e Dynamic checking of assertions O Deductive approach draw logical conclusions from e existing knowledge e the source code e test cases and test results O Experimental approach observe program execution e program state Software Quality 4 Debugging 2014 Martin Glinz 33 Example Program sample cf 4 1 O First hypothesis Program runs correctly Prediction Entering 11 14 yields 11 14 as result Experiment sample 11 14 Output 0 11 x Hypothesis is rejected Software Quality 4 Debugging 2014 Martin Glinz 34 Example Program sample cf 4 1 O Second hypothesis Program prints wrong variables Prediction a 0 11 a 1 14 but result is Output 0 11 Experiment Replace code for input and sorting by a 0 11 a 1 14 argc 3 Result Output 11 14 Y Hypothesis is rejected Software Quality 4 Debugging 2014 Martin Glinz 35 Static and dynamic analysis O Analyzing the co2. Create an as simple as possible test case that reproduces the reported problem Software Quality 4 Debugging 2014 Martin Glinz 16 Typical problems O Reproducing the environment in which the problem occurs O Reproducing the history trail may be necessary O For software errors reproduce a program run that causes the error this may include Input data Initial persistent data e User interaction interaction with neighboring systems e Time e Communication with other processes e Process threads e Random data Software Quality 4 Debugging 2014 Martin Glinz 17 Time dependent errors a case In early 1992 a company installed a new barrier gate control system in a couple of parking garages In the morning of September 12 1992 the operators of all these garages called the support line and reported the same problem the exit barriers didn t open anymore The date had been coded with two integers one for the year and one for the day of the year Unfortunately the programmer had chosen the type short int i e one byte for both variables September 12 was day 256 in that year Software Quality 4 Debugging 2014 Martin Glinz 18 4 1 Foundations 4 2 The Debugging Process 4 3 Reproducing Errors 4 4 Simplifying and Automating Test Cases 4 5 Techniques for Defect Localization 4 6 Defect Fixing Simplifying O Given a test case which reliably causes a reported error O Goal e Remove all irrelevant
3. NET A ih causes the error This set Sraa BID ESESUDSHO Re only contains the font Panra at canere SoS SS aa Hiragino Kaku Gothic Pro ABCDEFGHIJKLM abcdefghijklm 1234567890 4 ee OOO lt i gt Eil 225 Schriften HiraKakuPro W3 32 pt Software Quality 4 Debugging 2014 Martin Glinz 28 4 1 Foundations 4 2 The Debugging Process 4 3 Reproducing Errors 4 4 Simplifying and Automating Test Cases 4 5 Techniques for Defect Localization 4 6 Defect Fixing Overview O Create and test hypotheses O Static and dynamic program analysis e Control flow e Data flow O Analyze program states O Observe program execution stepping breakpointing O Dynamically check program assertions O Determine and isolate cause effect chains O Debugging by gut feeling Software Quality 4 Debugging 2014 Martin Glinz 30 Creating and testing hypotheses O The basis of systematic debugging O Principle Get insight through theory and experimentation 1 Create a hypothesis 2 Derive predictions from hypotheses 3 Verify predictions experimentally 4 If predictions and experiment results match e Correctness of hypothesis becomes more probable e Try to further confirm hypothesis fo eel Otherwise e Reject hypothesis e Create new or modified hypothesis continue with step 2 O Important record the track of all tested hypotheses 2014 Martin Glinz 31 Software Quality 4 Debugging Finding hypotheses
4. parts of the test case e Automate the simplified test case O In an optimally simplified test case all constituents are relevant i e removing anything from the case no longer produces the reported error O How to simplify Simplify environment e Reduce history trail Simplify inputs interactions Software Quality 4 Debugging 2014 Martin Glinz 20 Automating O The error provoking test case must be executed frequently in the debugging process e for finding simplifications e for testing hypotheses when systematically locating a defect Automation pays off O Test automation techniques gt Chapter 4 of this course Software Quality 4 Debugging 2014 Martin Glinz 21 simplify the environment O Determine which states or conditions in the system s environment are relevant and which ones aren t e Hardware and operating system e State of persistent data e Time e State of neighboring systems O Irrelevant states and conditions can be safely ignored O Goal minimize the effort for setting up the test environment in which the a test case produces the reported error O Means systematic trying Software Quality 4 Debugging 2014 Martin Glinz 22 simplify the error history O Can we reduce the number of steps required for provoking the error O Means systematic trying O Example Mozilla bug report no 24735 see above reports the following error provoking sequence of steps Start mozilla Go
5. to bugzilla mozilla org Select search for bug Print to file setting the bottom and right margins to 50 Once it s done printing do the exact same thing again on the same Tile Actually the following steps suffice to provoke the error Start mozilla Go to bugzilla mozilla org Select search tor bug Press Alt P Lett click on the Print button in the print dialog window Software Quality 4 Debugging 2014 Martin Glinz 23 Simplify inputs O Example Mozilla bug report no 24735 see above e The erroneous printing function uses the currently displayed web page as input e This page consists of 896 lines of html code O Which parts of this data cause the error and which ones are irrelevant O Means binary search Kernighan and Pike 1999 e Partition the set of input data into two halves e Test both halves individually e Recursively continue with that half which provokes the error Software Quality 4 Debugging 2014 Martin Glinz 24 simplify inputs 2 An example Zeller 2005 O Example Mozilla bug report no 24735 see above O Binary search yields a single fault provoking line of html code in twelve steps _ lt _ _ _ _ _ _ _ _ _ _ _ __ _ E_S Sli _CCC C C _ _ rti lt _ OC O C C C C CCC Y n OO O C O OOA OUON 12 lt SELECT NAME priority MULTIPLE SIZE 7 gt Software Quality 4 Debugging 2014 Martin Glinz 896 lines 448 lines
6. 224 lines 112 lines 112 lines 56 lines 1 line x KK EK KX Simplify inputs 3 O What to do if both halves don t provoke the error while the whole does lt SELECT NAME priority MULTIPLE SIZE 7 gt X lt SELECT NAME priority MULTIPLE SIZE 7 gt wY lt SELECT NAME priority MULTIPLE SIZE 7 gt wY O Instead of halves use smaller portions e g quarters lt SELECT NAME priority MULTIPLE SIZE 7 gt Y lt SELECT NAME priority MULTIPLE SIZE 7 gt X c lt SELECT NAME priority MULTIPLE SIZE 7 gt X lt SELECT NAME priority MULTIPLE SIZE 7 gt vY O Continue with eighths etc O Result lt SELECT gt x Software Quality 4 Debugging 2014 Martin Glinz 26 Automating the simplification O Simplification can be automated partially e In particular the technique of binary searching e Applicable for simplification of input data or interaction sequences O Example Zellers ddmin delta debugging algorithm Zeller 2005 Chapter 5 4 5 5 Software Quality 4 Debugging 2014 Martin Glinz 27 Another example Microsoft PowerPoint 2004 Version 11 0 on MacBook Pro with Mac OS 10 5 6 crashed during startup if the font Hiragino Kaku Gothic Pro was disabled in the font collection 000 Schriftsammlung em Q aay cea i BOI N K YAO Using interval bisection on aE Hoa 33 fe Ely the set of all fonts we can Be j find a minimal set of mro 4 RTbRC Aki ebIRWES deactivated fonts that SOS UURTRE
7. University of Zurich Department of Informatics Martin Glinz software Quality Chapter 4 Debugging 2014 Martin Glinz All rights reserved Making digital or hard copies of all or part of this work for educational non commercial use is permitted Using this material for any commercial purposes and or teaching is not permitted without prior written consent of the author Note that some images may be copyrighted by third parties 4 1 Foundations 4 2 The Debugging Process 4 3 Reproducing Errors 4 4 Simplifying and Automating Test Cases 4 5 Techniques for Defect Localization 4 6 Defect Fixing Terminology Debugging The process of finding and correcting a defect that causes an observed error Defect fault A faulty element in a program or other artifact Error A deviation of an observed result from the expected correct result O The term bug may denote a defect or an error O An error may be caused by a combination of multiple defects O The very same defect may manifest in more than one error O Program is meant in a comprehensive way may be a single method or a component or a complete system Software Quality 4 Debugging 2014 Martin Glinz Causes and Effects O Typically a defect e does not immediately lead to an error that can be observed but to faulty program states that propagate e and eventually manifest as observable errors O The main task of debugging is id
8. and then switch to systematic debugging O Suggested procedure e For a strictly limited time debug by intuition e f success Eureka else stop and start systematic debugging Software Quality 4 Debugging 2014 Martin Glinz 47 4 1 Foundations 4 2 The Debugging Process 4 3 Reproducing Errors 4 4 Simplifying and Automating Test Cases 4 5 Techniques for Defect Localization 4 6 Defect Fixing Fixing a localized defect If a defect has been located O Estimate severity of defect O Determine what and how much has to be fixed O Estimate impact on other parts of the system O Make the required modifications to the code and or the documentation carefully and systematically O Avoid quick and dirty patching of code Software Quality 4 Debugging 2014 Martin Glinz 49 Check effectiveness of problem resolution O Make sure that the reported problem no longer exists In case of software errors e Inspect the modified code and documentation e Test the modified units using the error provoking test case s e by writing more unit test cases O Check for unexpected side effects e Adapt the regression test suite to the modified code e Perform a regression test O Create a new configuration release Software Quality 4 Debugging 2014 Martin Glinz 50 Learning from the fixed defect Defects are typically due to mistakes by humans O Try to determine guess the reasons why somebody made the mistake s that led to the def
9. defect that causes the error e Create and test hypotheses e Observe program states e Check the validity of assertions in the program e Isolate cause effect chains O Fix the identified defect s Software Quality 4 Debugging 2014 Martin Glinz 12 Checking the effectiveness of the fix O Make sure that the defect has been fixed e Re run the test case s that resulted in errors e Everything ok now O Make sure that the fix did not create any new defects e Run your regression test suite e No new problems found Software Quality 4 Debugging 2014 Martin Glinz Required infrastructure O Problem reporting infrastructure e Process for handling problem reports e Tool for problem report administration and tracking For example Bugzilla O Configuration management system for software artifacts Software Quality 4 Debugging 2014 Martin Glinz 14 4 1 Foundations 4 2 The Debugging Process 4 3 Reproducing Errors 4 4 Simplifying and Automating Test Cases 4 5 Techniques for Defect Localization 4 6 Defect Fixing A sample bug report Example Mozilla bug report no 24735 from 1999 Start mozilla Go to bugzilla mozilla org Select search for bug Print to file setting the bottom and right margins to 50 I use the file var tmp netscape ps Once it s done printing do the exact same thing again on the same file var tmp netscape ps This causes the browser to crash with a segfault Zeller 2005 p 55 Goal
10. ect O Investigate if there are any similar defects in the source code that stem from the same kind of mistake O Are there any constructive means to avoid such defects in the future e g by e changing a process e training people Software Quality 4 Debugging 2014 Martin Glinz 51 References S C McConnell 1993 Code Complete A Practical Handbook of Software Construction Redmond Microsoft Press B W Kernighan R Pike 1999 The Practice of Programming Reading Mass Addison Wesley M Weiser 1992 Programmers Use Slices When Debugging Communications of the ACM 25 7 446 452 A Zeller 2002 Isolating Cause Effect Chains from Computer Programs Proceedings of the 10th ACM SIGSOFT Symposium on the Foundations of Software Engineering Charleston South Carolina 1 10 A Zeller 2005 Why Programs Fail A Guide to Systematic Debugging Amsterdam Morgan Kaufmann and Heidelberg dpunkt Bugzilla http Awww bugzilla org Logging Services http logging apache org Software Quality 4 Debugging 2014 Martin Glinz 52
11. ence in the code is 1 in line 36 O This is a minimal cause of the error Software Quality 4 Debugging 2014 Martin Glinz 44 Identifying and isolating cause effect chains O The immediate cause of an error normally is not a defect but an erroneous program state eventually caused by a defect e Identify cause effect chains and isolate them from the irrelevant rest of the program O Time consuming Requires creation and test of many hypotheses O Systematic procedure needed O Automatable Zeller s Delta Debugging algorithm Zeller 2002 Software Quality 4 Debugging 2014 Martin Glinz 45 Isolating causes with Delta Debugging O Difference between isolation and simplification e Simplification Find a minimal error provoking test case e Isolation Find an error provoking and an error free test case with a minimal difference O Example Isolation of minimal error cause in this input Error provoking 5 6 201 x Reduce cranoeus 5 6 2 1 Y case 53672 1 Y 5672 17 Y Extend error free Error free empty input Y case Software Quality 4 Debugging 2014 Martin Glinz 46 Debugging by gut feeling O To some extent experienced software engineers develop an ability to smell the cause of an error O In many cases debugging by intuition is faster than any systematic debugging procedure O Problem e We need to stop intuitive debugging at the right time when it does not succeed e
12. entifying reconstructing the cause effect chain from a defect to an observable error Software Quality 4 Debugging 2014 Martin Glinz Where defects occur O Classic defect is a coding error caused by a human mistake O Alternatively e Defects in other artifacts requirements specification system architecture system design user manual e Defects in the data e Defects in processes e Human mistakes when using or operating a system O Some defects are not local but affect a complete system or sub system Software Quality 4 Debugging 2014 Martin Glinz 5 Example A simple sorting problem Zeller 2005 Name sample Author Andreas Zeller Language C Call sample arg arg arg Precondition arg arg arg are integers n E IN Postcondition The arguments appear in ascending order on the standard output device Executing sample with test data sample 9 7 8 sample 11 14 Output 7 8 9 Output _ Software Quality 4 Debugging 2014 Martin Glinz 6 Program sample The code sample c Sample C program to be debugged include lt stdio h gt include lt stdlib h gt static void shell sort int a int size int i j int h 1 do h h 3 1 while h lt size do h 3 for i h i lt size i int v a i for j i j gt h amp amp a j h gt v a j a j h if i j a j v while h 1 Software Q
13. ng framework such as LOG4J Logging Services e Using a debugger e Compile program in debug mode e Halt execution at critical points by setting breakpoints e Inspect current variable values Software Quality 4 Debugging 2014 Martin Glinz 38 Example Program sample cf 4 1 O Third hypothesis Sorting procedure called with wrong parameters Prediction Values in array a and or value of argc wrong Experiment Prior to the call of shell_sort we instrument the source code with printf Parameters of shell sort for i 0 i lt argc i printf Sd a i printf d argc printf n Result Parameters of shell sort 111403 X Hypothesis is confirmed O Alternatively we could have used a debugger Software Quality 4 Debugging 2014 Martin Glinz 39 Observe program execution Using a debugger we can O Stepwise execute a program or halt it at breakpoints e Compare expected and actual control flow e Inspect parts of system state where appropriate O Observe variable definition modification and use Software Quality 4 Debugging 2014 Martin Glinz 40 Checking assertions O Specifying contracts for classes and methods with assertions e Preconditions e Postconditions e Invariants Formally specified contracts can be checked dynamically by a suitable runtime system O When an assertion is violated analyze the program state Software Quality 4 Debugging 2014 Martin Glinz 41 Cause
14. ntrol flow and the data flow of a program see Chapter 3 on data flow testing and Chapter 12 of my Software Engineering course O Static Analysis e Yields the potentially possible control and data flows e No program execution required e Independent of any concrete test cases O Dynamic Analysis e Analyzes a concrete program run based on a test case e Yields actual control and data flows for this run Software Quality 4 Debugging 2014 Martin Glinz 36 Example static vs dynamic program slicing int main int main int main int a b sum mul int a b sum mul int a b sum mul sum 0 sum 0 sum 0 mul 1 mul 1 mul 1 a read a read a read b read b read b read while a lt b while a lt b while a lt b sum sum a sum sum a sum sum a mul mul a mul mul a mul mul a a a L a a l a aitl write sum write sum write sum write mul write mul write mul Sample program Static slice of mul in Dynamic slice of mul in line 13 line 13 with a 5 b 2 Software Quality 4 Debugging 2014 Martin Glinz 37 Analysis of program states O The problem a defect typically e leads to a sequence of erroneous states e that eventually manifest in observable errors O Check suspicious program states e Instrumentation of the code e Record variable values e Print or log variable values maybe using a loggi
15. s and effects O An observation e In the decade of 1950 to1960 the decline of the population of storks in Europe is strongly correlated with the increasing number of tarmac roads O Question e Is the increasing number of tarmac roads the a cause for the disappearance of storks O Testing for causality a is a cause for b iff e b occurs if a has occurred previously e b does not occur if a has not occurred previously e All other variables are kept constant Software Quality 4 Debugging 2014 Martin Glinz 42 Causes and effects 2 O Experimental proof of or evidence for causality e Generally rather difficult Problem of controlled experiments e For debugging it is doable Controlled environment e Test case reproducible O In debugging a cause for an error f can be viewed as the difference between e atest case exhibiting the error f 1 e atest case that runs correctly 2 O Again we look for a minimal cause Search a minimal difference between 1 and 2 Software Quality 4 Debugging 2014 Martin Glinz 43 Example Program sample cf 4 1 O Fourth hypothesis shell sort should be called with argc 1 instead of argc Prediction Result is correct Experiment Execute with modified source code or modify state of running program with a debugger Result Output 11 14 Y Hypothesis is confirmed O From the first hypothesis we know that calling shell_sort with argc leads to an error O The differ
16. uality 4 Debugging 2014 Martin Glinz Program sample The code 2 int main int argc char argv int a i int malloc argc 1 sizeof int i 0 i lt argc 1 itt a i atoi argv i 1 shell sort a argc printf Output for i 0 i lt argc 1 i printf d a i printf n free a return 0 Software Quality 4 Debugging 2014 Martin Glinz What now Observation There are input data for which sample computes a wrong result Question O How do we find the defect in the code that causes this error O Is there a way of systematically searching for a defect Software Quality 4 Debugging 2014 Martin Glinz 4 1 Foundations 4 2 The Debugging Process 4 3 Reproducing Errors 4 4 Simplifying and Automating Test Cases 4 5 Techniques for Defect Localization 4 6 Defect Fixing The main steps of the debugging process O Describe the problem precisely e Sometimes this alone reveals the source of the problem O Is the problem a software error If yes e Perform classic debugging If no e Search and fix the problem elsewhere e g Defects in user manuals Faulty business processes Training deficits O Check the effectiveness of the fix Software Quality 4 Debugging 2014 Martin Glinz 11 The classic software debugging process O Reproduce the error O Simplify and if possible automate the test case that produces the error O Localize the
Download Pdf Manuals
Related Search