The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 3

Do you have a test case in which all sides are zero (0,0,0)?12. Do you have at least one test case specifying noninteger values (such as 2.5,3.5,5.5)?13. Do you have at least one test case specifying the wrong number of values (two rather than three integers, for example)?14. For each test case did you specify the expected output fromthe program in addition to the input values?Of course, a set of test cases that satisfies these conditions does notguarantee that all possible errors would be found, but since questions1 through 13 represent errors that actually have occurred in differentversions of this program, an adequate test of this program shouldexpose at least these errors.Now, before you become concerned about your own score, consider this: In our experience, highly qualified professional programmers score, on the average, only 7.8 out of a possible 14.

If you’vedone better, congratulations; if not, we’ll try to help.The point of the exercise is to illustrate that the testing of even atrivial program such as this is not an easy task. And if this is true, consider the difficulty of testing a 100,000-statement air traffic control4The Art of Software Testingsystem, a compiler, or even a mundane payroll program. Testing alsobecomes more difficult with the object-oriented languages such asJava and C++. For example, your test cases for applications built withthese languages must expose errors associated with object instantiation and memory management.It might seem, from working with this example, that thoroughlytesting a complex, real-world program would be impossible. Not so!Although the task can be daunting, adequate program testing is avery necessary—and achievable—part of software development, asyou will learn in this book.CHAPTER 2The Psychology andEconomics ofProgram TestingSoftware testing is a technical task,but it also involves some important considerations of economics andhuman psychology.In an ideal world, we would want to test every possible permutation of a program.

In most cases, however, this simply is not possible.Even a seemingly simple program can have hundreds or thousands ofpossible input and output combinations. Creating test cases for all ofthese possibilities is impractical. Complete testing of a complex application would take too long and require too many human resources tobe economically feasible.In addition, the software tester needs the proper attitude (perhaps“vision” is a better word) to successfully test a software application.In some cases, the tester’s attitude may be more important than theactual process itself. Therefore, we will start our discussion of software testing with these issues before we delve into the more technical nature of the topic.The Psychology of TestingOne of the primary causes of poor program testing is the fact thatmost programmers begin with a false definition of the term.

Theymight say:• “Testing is the process of demonstrating that errors are notpresent.”56The Art of Software Testingor• “The purpose of testing is to show that a program performs itsintended functions correctly.”or• “Testing is the process of establishing confidence that a programdoes what it is supposed to do.”These definitions are upside-down.When you test a program, you want to add some value to it.Adding value through testing means raising the quality or reliabilityof the program. Raising the reliability of the program means findingand removing errors.Therefore, don’t test a program to show that it works; rather, youshould start with the assumption that the program contains errors(a valid assumption for almost any program) and then test the program to find as many of the errors as possible.Thus, a more appropriate definition is this:Testing is the process of executing a programwith the intent of finding errors.Although this may sound like a game of subtle semantics, it’s really animportant distinction.

Understanding the true definition of softwaretesting can make a profound difference in the success of your efforts.Human beings tend to be highly goal-oriented, and establishingthe proper goal has an important psychological effect. If our goal isto demonstrate that a program has no errors, then we will subconsciously be steered toward this goal; that is, we tend to select testdata that have a low probability of causing the program to fail. Onthe other hand, if our goal is to demonstrate that a program haserrors, our test data will have a higher probability of finding errors.The latter approach will add more value to the program than theformer.This definition of testing has myriad implications, many of whichare scattered throughout this book.

For instance, it implies that testing is a destructive process, even a sadistic process, which explainsThe Psychology and Economics of Program Testing7why most people find it difficult. That may go against our grain; withgood fortune, most of us have a constructive, rather than a destructive, outlook on life. Most people are inclined toward making objectsrather than ripping them apart. The definition also has implicationsfor how test cases (test data) should be designed and who should andwho should not test a given program.Another way of reinforcing the proper definition of testing is toanalyze the use of the words “successful” and “unsuccessful”—inparticular, their use by project managers in categorizing the results oftest cases.

Most project managers call a test case that did not find anerror a “successful test run,” whereas a test that discovers a new erroris usually called “unsuccessful.”Once again, this is upside-down. “Unsuccessful” denotes something undesirable or disappointing. To our way of thinking, a wellconstructed and -executed test of a piece of software is successfulwhen it finds errors that can be fixed. And that same test is also successful when it eventually establishes that there are no more errors tobe found. The only unsuccessful test is one that does not properlyexamine the software and, in the majority of cases, a test that foundno errors would likely be considered unsuccessful, since the conceptof a program without errors is basically unrealistic.A test case that finds a new error can hardly be considered unsuccessful; rather, it has proven to be a valuable investment.

An unsuccessful test case is one that causes a program to produce the correctresult without finding any errors.Consider the analogy of a person visiting a doctor because of anoverall feeling of malaise. If the doctor runs some laboratory tests thatdo not locate the problem, we do not call the laboratory tests “successful”; they were unsuccessful tests in that the patient’s net worth hasbeen reduced by the expensive laboratory fees, the patient is still ill,and the patient may question the doctor’s ability as a diagnostician.However, if a laboratory test determines that the patient has a pepticulcer, the test is successful because the doctor can now begin theappropriate treatment. Hence, the medical profession seems to usethese words in the proper sense.

The analogy, of course, is that weshould think of the program, as we begin testing it, as the sick patient.8The Art of Software TestingA second problem with such definitions as “testing is the processof demonstrating that errors are not present” is that such a goal isimpossible to achieve for virtually all programs, even trivial programs.Again, psychological studies tell us that people perform poorlywhen they set out on a task that they know to be infeasible orimpossible. For instance, if you were instructed to solve the crossword puzzle in the Sunday New York Times in 15 minutes, we wouldprobably observe little, if any, progress after 10 minutes becausemost of us would be resigned to the fact that the task seems impossible.

If you were asked for a solution in four hours, however, wecould reasonably expect to see more progress in the initial 10 minutes. Defining program testing as the process of uncovering errorsin a program makes it a feasible task, thus overcoming this psychological problem.A third problem with the common definitions such as “testing isthe process of demonstrating that a program does what it is supposedto do” is that programs that do what they are supposed to do still cancontain errors. That is, an error is clearly present if a program doesnot do what it is supposed to do, but errors are also present if a program does what it is not supposed to do. Consider the triangle program ofChapter 1. Even if we could demonstrate that the program correctlydistinguishes among all scalene, isosceles, and equilateral triangles,the program still would be in error if it does something it is not supposed to do (such as representing 1,2,3 as a scalene triangle or sayingthat 0,0,0 represents an equilateral triangle).

We are more likely todiscover the latter class of errors if we view program testing as theprocess of finding errors than if we view it as the process of showingthat a program does what it is supposed to do.To summarize, program testing is more properly viewed as thedestructive process of trying to find the errors (whose presence isassumed) in a program. A successful test case is one that furthersprogress in this direction by causing the program to fail. Of course,you eventually want to use program testing to establish some degreeof confidence that a program does what it is supposed to do and doesnot do what it is not supposed to do, but this purpose is best achievedby a diligent exploration for errors.The Psychology and Economics of Program Testing9Consider someone approaching you with the claim that “my program is perfect” (error free).