The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 29

Of the 38 test166The Art of Software Testingcases identified by the process of cause-effect graphing, we start byrunning four test cases. As part of the process of establishing inputconditions, we will initialize memory that the first, fifth, ninth, . . . ,words have the value 000; the second, sixth, . . . , words have thevalue 4444; the third, seventh, . . .

, words have the value 8888; and thefourth, eighth, . . . , words have the value CCCC. That is, each memory word is initialized to the low-order hexadecimal digit in theaddress of the first byte of the word (the values of locations 23FC,23FD, 23FE, and 23FF are C).The test cases, their expected output, and the actual output afterthe test are shown in Figure 7.5.Obviously, we have some problems, since none of the test casesapparently produced the expected results (all were successful), butlet’s start by debugging the error associated with the first test case.The command indicates that, starting at location 0 (the default),E locations (14 in decimal) are to be displayed. (Recall that the specification stated that all output will contain four words or 16 bytes perline.)Figure 7.5Test case results from the DISPLAY command.Debugging167Enumerating the possible causes for the unexpected error message,we might get1.

The program does not accept the word DISPLAY.2. The program does not accept the period.3. The program does not allow a default as a first operand; itexpects a storage address to precede the period.4. The program does not allow an E as a valid byte count.The next step is to try to eliminate the causes.

If all are eliminated,we must retreat and expand the list. If more than one remain, wemight want to examine additional test cases to arrive at a single errorhypothesis, or proceed with the most probable cause. Since we haveother test cases at hand, we see that the second test case in Figure 7.5seems to eliminate the first hypothesis, and the third test case,although it produced an incorrect result, seems to eliminate the second and third hypotheses.The next step is to refine the fourth hypothesis.

It seems specificenough, but intuition might tell us that there is more to it than meetsthe eye; it sounds like an instance of a more general error. We mightcontend, then, that the program does not recognize the special hexadecimal characters A–F. This absence of such characters in the othertest cases makes this sound like a viable explanation.Rather than jumping to a conclusion, however, we should firstconsider all of the available information.

The fourth test case mightrepresent a totally different error, or it might provide a clue about thecurrent error. Given that the highest valid address in our system is7FFF, how could the fourth test case be displaying an area that appearsto be nonexistent? The fact that the displayed values are our initializedvalues and not garbage might lead to the supposition that this command is somehow displaying something in the range 0–7FFF. One ideathat may arise is that this could occur if the program is treating theoperands in the command as decimal values rather than hexadecimal asstated in the specification. This is borne out by the third test case;rather than displaying 32 bytes of memory, the next increment above11 in hexadecimal (17 in base 10), it displays 16 bytes of memory,168The Art of Software Testingwhich is consistent with our hypothesis that the “11” is being treatedas a base-10 value. Hence, the refined hypothesis is that the programis treating the byte count as storage address operands, and the storageaddresses on the output listing as decimal values.The last step is to prove this hypothesis.

Looking at the fourth testcase, if 8000 is interpreted as a decimal number, the correspondingbase-16 value is 1F40, which would lead to the output shown. As further proof, examine the second test case. The output is incorrect, butif 21 and 29 are treated as decimal numbers, the locations of storageaddresses 15–1D would be displayed; this is consistent with the erroneous result of the test case. Hence, we have almost certainly locatedthe error; the program is assuming that the operands are decimal values and is printing the memory addresses as decimal values, whichis inconsistent with the specification. Moreover, this error seems tobe the cause of the erroneous results of all four test cases.

A littlethought has led to the error, and it also solved three other problemsthat, at first glance, appear to be unrelated.Note that the error probably manifests itself at two locations in theprogram: the part that interprets the input command and the part thatprints memory addresses on the output listing.As an aside, this error, likely caused by a misunderstanding of thespecification, reinforces the suggestion that a programmer should notattempt to test his or her own program. If the programmer who created this error is also designing the test cases, he or she likely willmake the same mistake while writing the test cases. In other words,the programmer’s expected outputs would not be those of Figure7.5; they would be the outputs calculated under the assumption thatthe operands are decimal values.

Therefore, this fundamental errorprobably would go unnoticed.Debugging by BacktrackingAn effective method for locating errors in small programs is to backtrack the incorrect results through the logic of the program until youDebugging169find the point where the logic went astray. In other words, start atthe point where the program gives the incorrect result—such aswhere incorrect data were printed.

At this point you deduce fromthe observed output what the values of the program’s variables musthave been. By performing a mental reverse execution of the program from this point and repeatedly using the process of “if this wasthe state of the program at this point, then this must have been thestate of the program up here,” you can quickly pinpoint the error.With this process you’re looking for the location in the programbetween the point where the state of the program was what wasexpected and the first point where the state of the program was whatwas not expected.Debugging by TestingThe last “thinking type” debugging method is the use of test cases.This probably sounds a bit peculiar since the beginning of this chapter distinguishes debugging from testing.

However, consider twotypes of test cases: test cases for testing, where the purpose of the testcases is to expose a previously undetected error, and test cases fordebugging, where the purpose is to provide information useful inlocating a suspected error. The difference between the two is that testcases for testing tend to be “fat” because you are trying to covermany conditions in a small number of test cases.

Test cases for debugging, on the other hand, are “slim” since you want to cover only asingle condition or a few conditions in each test case.In other words, after a symptom of a suspected error is discovered,you write variants of the original test case to attempt to pinpoint theerror. Actually, this method is not an entirely separate method; itoften is used in conjunction with the induction method to obtaininformation needed to generate a hypothesis and/or to prove ahypothesis. It also is used with the deduction method to eliminatesuspected causes, refine the remaining hypothesis, and/or prove ahypothesis.170The Art of Software TestingDebugging PrinciplesIn this section, we want to discuss a set of debugging principles thatare psychological in nature.

As was the case for the testing principlesin Chapter 2, many of these debugging principles are intuitivelyobvious, yet they are often forgotten or overlooked. Since debuggingis a two-part process—locating an error and then repairing it—twosets of principles are discussed.Error-Locating PrinciplesThinkAs implied in the previous section, debugging is a problem-solvingprocess. The most effective method of debugging is a mental analysisof the information associated with the error’s symptoms.

An efficientprogram debugger should be able to pinpoint most errors withoutgoing near a computer.If You Reach an Impasse, Sleep on ItThe human subconscious is a potent problem solver. What we oftenrefer to as inspiration is simply the subconscious mind working on aproblem when the conscious mind is working on something elsesuch as eating, walking, or watching a movie. If you cannot locate anerror in a reasonable amount of time (perhaps 30 minutes for a smallprogram, several hours for a larger one), drop it and work on something else, since your thinking efficiency is about to collapse anyway.After forgetting about the problem for a while, your subconsciousmind will have solved the problem, or your conscious mind will beclear for a fresh examination of the symptoms.If You Reach an Impasse, Describe the Problemto Someone ElseTalking about the problem with someone else may help you discoversomething new.

In fact, often simply by describing the problem to aDebugging171good listener, you will suddenly see the solution without any assistance from the listener.Use Debugging Tools Only as a Second ResortUse debugging tools after you’ve tried other methods, and then onlyas an adjunct to, not a substitute for, thinking. As noted earlier in thischapter, debugging tools, such as dumps and traces, represent a haphazard approach to debugging.