The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 10

This assumption is invalid. Human testing techniquesare very effective at revealing errors. In fact, most programming projects should include the following human testing techniques:••••Code inspections using checklistsGroup walkthroughsDesk checkingPeer reviewsCHAPTER 4Test-Case DesignMoving beyond the psychologicalissues discussed in Chapter 2, the most important consideration inprogram testing is the design and creation of effective test cases.Testing, however creative and seemingly complete, cannot guarantee the absence of all errors. Test-case design is so important becausecomplete testing is impossible; a test of any program must be necessarily incomplete. The obvious strategy, then, is to try to make testsas complete as possible.Given constraints on time and cost, the key issue of testingbecomesWhat subset of all possible test cases has the highestprobability of detecting the most errors?The study of test-case-design methodologies supplies answers to thisquestion.In general, the least effective methodology of all is random-inputtesting—the process of testing a program by selecting, at random,some subset of all possible input values.

In terms of the likelihood ofdetecting the most errors, a randomly selected collection of test caseshas little chance of being an optimal, or close to optimal, subset. Inthis chapter we want to develop a set of thought processes that let youselect test data more intelligently.Chapter 2 showed that exhaustive black-box and white-box testing are, in general, impossible, but suggested that a reasonable testingstrategy might be elements of both. This is the strategy developed inthis chapter. You can develop a reasonably rigorous test by using certain black-box-oriented test-case-design methodologies and then4344The Art of Software Testingsupplementing these test cases by examining the logic of the program, using white-box methods.The methodologies discussed in this chapter are listed as follows.Black BoxEquivalence partitioningBoundary-value analysisCause-effect graphingError guessingWhite BoxStatement coverageDecision coverageCondition coverageDecision-condition coverageMultiple-condition coverageAlthough the methods will be discussed separately, we recommendthat you use a combination of most, if not all, of the methods to designa rigorous test of a program, since each method has distinct strengthsand weaknesses.

One method may find errors another method overlooks, for example.Nobody ever promised that software testing would be easy. Toquote an old sage, “If you thought designing and coding that program was hard, you ain’t seen nothing yet.”The recommended procedure is to develop test cases using theblack-box methods and then develop supplementary test cases asnecessary with white-box methods. We’ll discuss the more widelyknown white-box methods first.White-Box TestingLogic-Coverage TestingWhite-box testing is concerned with the degree to which test casesexercise or cover the logic (source code) of the program.

As we sawin Chapter 2, the ultimate white-box test is the execution of everypath in the program, but complete path testing is not a realistic goalfor a program with loops.If you back completely away from path testing, it may seem that aworthy goal would be to execute every statement in the program atleast once. Unfortunately, this is a weak criterion for a reasonableTest-Case Design45white-box test. This concept is illustrated in Figure 4.1. Assume thatFigure 4.1 represents a small program to be tested. The equivalentJava code snippet follows:public void foo(int a, int b, int x) {if (a>1 && b==0){x=x/a;}if (a==2 || x>1){x=x+1;}}You could execute every statement by writing a single test case thattraverses path ace.

That is, by setting A=2, B=0, and X=3 at point a, everystatement would be executed once (actually, X could be assigned anyvalue).Unfortunately, this criterion is a rather poor one. For instance, perhaps the first decision should be an or rather than an and. If so, thiserror would go undetected. Perhaps the second decision should havestated X>0; this error would not be detected. Also, there is a paththrough the program in which X goes unchanged (the path abd). If thiswere an error, it would go undetected. In other words, the statementcoverage criterion is so weak that it generally is useless.A stronger logic-coverage criterion is known as decision coverage orbranch coverage. This criterion states that you must write enough testcases that each decision has a true and a false outcome at least once.

Inother words, each branch direction must be traversed at least once.Examples of branch or decision statements are switch, do-while, andif-else statements. Multiway GOTO statements qualify in some programming languages such as FORTRAN.Decision coverage usually can satisfy statement coverage. Sinceevery statement is on some subpath emanating either from a branchstatement or from the entry point of the program, every statementmust be executed if every branch direction is executed.

However,there are at least three exceptions:46The Art of Software Testing• Programs with no decisions.• Programs or subroutines/methods with multiple entry points. Agiven statement might be executed only if the program isentered at a particular entry point.• Statements within ON-units. Traversing every branch directionwill not necessarily cause all ON-units to be executed.Since we have deemed statement coverage to be a necessary condition, decision coverage, a seemingly better criterion, should be definedto include statement coverage. Hence, decision coverage requires thateach decision have a true and a false outcome, and that each statementbe executed at least once.

An alternative and easier way of expressing itis that each decision has a true and a false outcome, and that each pointof entry (including ON-units) be invoked at least once.This discussion considers only two-way decisions or branches andhas to be modified for programs that contain multiway decisions.Examples are Java programs containing select (case) statements,FORTRAN programs containing arithmetic (three-way) IF statements or computed or arithmetic GOTO statements, and COBOLprograms containing altered GOTO statements or GO-TO-DEPENDING-ONstatements.

For such programs, the criterion is exercising each possible outcome of all decisions at least once and invoking each point ofentry to the program or subroutine at least once.In Figure 4.1, decision coverage can be met by two test cases covering paths ace and abd or, alternatively, acd and abe.

If we choose thelatter alternative, the two test-case inputs are A = 3, B = 0, X = 3 andA = 2, B = 1, and X = 1.Decision coverage is a stronger criterion than statement coverage,but it still is rather weak. For instance, there is only a 50 percentchance that we would explore the path where X is not changed (i.e.,only if we chose the former alternative). If the second decision werein error (if it should have said X<1 instead of X>1), the mistake wouldnot be detected by the two test cases in the previous example.A criterion that is sometimes stronger than decision coverage iscondition coverage.

In this case, you write enough test cases to ensurethat each condition in a decision takes on all possible outcomes atTest-Case Design47Figure 4.1A small program to be tested.least once. Since, as with decision coverage, this does not always leadto the execution of each statement, an addition to the criterion is thateach point of entry to the program or subroutine, as well as ONunits, be invoked at least once. For instance, the branching statementDO K=0 to 50 WHILE (J+K<QUEST)contains two conditions: is K less than or equal to 50, and is J+K lessthan QUEST? Hence, test cases would be required for the situations48The Art of Software TestingK<=50, K>50(to reach the last iteration of the loop),J+K<QUEST,andJ+K>=QUEST.Figure 4.1 has four conditions: A>1, B=0, A=2, and X>1.

Hence,enough test cases are needed to force the situations where A>1, A<=1,B=0, and B<>0 are present at point a and where A=2, A<>2, X>1, and X<=1are present at point b. A sufficient number of test cases satisfying thecriterion, and the paths traversed by each, are1. A=2, B=0, X=42. A=1, B=1, X=1aceadbNote that, although the same number of test cases was generatedfor this example, condition coverage usually is superior to decisioncoverage in that it may (but does not always) cause every individualcondition in a decision to be executed with both outcomes, whereasdecision coverage does not. For instance, in the same branchingstatementDO K=0 to 50 WHILE (J+K<QUEST)is a two-way branch (execute the loop body or skip it).