The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 11

If you areusing decision testing, the criterion can be satisfied by letting theloop run from K=0 to 51, without ever exploring the circumstance where theWHILE clause becomes false. With the condition criterion, however, a testcase would be needed to generate a false outcome for the conditionsJ+K<QUEST.Although the condition-coverage criterion appears, at first glance,to satisfy the decision-coverage criterion, it does not always do so. Ifthe decision IF (A&B) is being tested, the condition-coverage criterion would let you write two test cases—A is true, B is false, and A isfalse, B is true—but this would not cause the THEN clause of the IF toexecute.

The condition-coverage tests for the earlier example cov-Test-Case Design49ered all decision outcomes, but this was only by chance. For instance,two alternative test cases1. A=1, B=0, X=32. A=2, B=1, X=1cover all condition outcomes, but they cover only two of the fourdecision outcomes (both of them cover path abe and, hence, do notexercise the true outcome of the first decision and the false outcomeof the second decision).The obvious way out of this dilemma is a criterion called decision/condition coverage. It requires sufficient test cases that each condition in a decision takes on all possible outcomes at least once, eachdecision takes on all possible outcomes at least once, and each pointof entry is invoked at least once.A weakness with decision/condition coverage is that, although itmay appear to exercise all outcomes of all conditions, it frequently doesnot because certain conditions mask other conditions.

To see this,examine Figure 4.2. The flowchart in Figure 4.2 is the way a compilerwould generate machine code for the program in Figure 4.1. Themulticondition decisions in the source program have been broken intoindividual decisions and branches because most machines do not havea single instruction that makes multicondition decisions. A more thorough test coverage, then, appears to be the exercising of all possibleoutcomes of each primitive decision. The two previous decisioncoverage test cases do not accomplish this; they fail to exercise the falseoutcome of decision H and the true outcome of decision K.The reason, as shown in Figure 4.2, is that results of conditions inand and or expressions can mask or block the evaluation of other conditions. For instance, if an and condition is false, none of the subsequent conditions in the expression need be evaluated. Likewise if anor condition is true, none of the subsequent conditions need be evaluated.

Hence, errors in logical expressions are not necessarily revealedby the condition-coverage and decision/condition-coverage criteria.A criterion that covers this problem, and then some, is multiple-50The Art of Software TestingFigure 4.2Machine code for the program in Figure 4.1.condition coverage. This criterion requires that you write sufficient testcases that all possible combinations of condition outcomes in eachdecision, and all points of entry, are invoked at least once. For instance,consider the following sequence of pseudocode.NOTFOUND=TRUE;DO I=1 to TABSIZE WHILE (NOTFOUND); /*SEARCH TABLE*/...searching logic...;ENDTest-Case Design51The four situations to be tested are:1. I<=TABSIZE and NOTFOUND is true.2.

I<=TABSIZE and NOTFOUND is false (finding the entry before hitting the end of the table).3. I>TABSIZE and NOTFOUND is true (hitting the end of the tablewithout finding the entry).4. I>TABSIZE and NOTFOUND is false (the entry is the last one inthe table).It should be easy to see that a set of test cases satisfying the multiplecondition criterion also satisfies the decision-coverage, conditioncoverage, and decision/condition-coverage criteria.Returning to Figure 4.1, test cases must cover eight combinations:1.

A>1, B=02. A>1, B<>03. A<=1, B=04. A<=1, B<>05. A=2, X>16. A=2, X<=17. A<>2, X>18. A<>2, X<=1Note, as was the case earlier, that cases 5 through 8 express values atthe point of the second if statement. Since x may be altered abovethis if statement, the values needed at this if statement must bebacked up through the logic to find the corresponding input values.These combinations to be tested do not necessarily imply thateight test cases are needed. In fact, they can be covered by four testcases.

The test-case input values, and the combinations they cover,are as follows:A=2, B=0, X=4A=2, B=1, X=1A=1, B=0, X=2A=1, B=1, X=1Covers 1, 5Covers 2, 6Covers 3, 7Covers 4, 8The fact that there are four test cases and four distinct paths in Figure4.1 is just coincidence. In fact, these four test cases do not cover every52The Art of Software Testingpath; they miss the path acd. For instance, you would need eight testcases for the following decision:if(x==y && length(z)==0 && FLAG) {j=1;elsei=1;}although it contains only two paths.

In the case of loops, the numberof test cases required by the multiple-condition criterion is normallymuch less than the number of paths.In summary, for programs containing only one condition per decision, a minimum test criterion is a sufficient number of test cases to(1) evoke all outcomes of each decision at least once and (2) invokeeach point of entry (such as entry point or ON-unit) at least once, toensure that all statements are executed at least once. For programscontaining decisions having multiple conditions, the minimum criterion is a sufficient number of test cases to evoke all possible combinations of condition outcomes in each decision, and all points of entryto the program, at least once.

(The word “possible” is inserted becausesome combinations may be found to be impossible to create.)Equivalence PartitioningChapter 2 described a good test case as one that has a reasonableprobability of finding an error, and it also discussed the fact that anexhaustive-input test of a program is impossible.

Hence, in testing aprogram, you are limited to trying a small subset of all possible inputs.Of course, then, you want to select the right subset, the subset withthe highest probability of finding the most errors.One way of locating this subset is to realize that a well-selected testcase also should have two other properties:1. It reduces, by more than a count of one, the number of othertest cases that must be developed to achieve some predefinedgoal of “reasonable” testing.Test-Case Design532. It covers a large set of other possible test cases. That is, it tellsus something about the presence or absence of errors overand above this specific set of input values.These two properties, although they appear to be similar, describetwo distinct considerations. The first implies that each test caseshould invoke as many different input considerations as possible tominimize the total number of test cases necessary.

The second impliesthat you should try to partition the input domain of a program intoa finite number of equivalence classes such that you can reasonablyassume (but, of course, not be absolutely sure) that a test of a representative value of each class is equivalent to a test of any other value.That is, if one test case in an equivalence class detects an error, allother test cases in the equivalence class would be expected to find thesame error.

Conversely, if a test case did not detect an error, wewould expect that no other test cases in the equivalence class wouldfall within another equivalence class, since equivalence classes mayoverlap one another.These two considerations form a black-box methodology known asequivalence partitioning. The second consideration is used to develop a setof “interesting” conditions to be tested. The first consideration is thenused to develop a minimal set of test cases covering these conditions.An example of an equivalence class in the triangle program ofChapter 1 is the set “three equal-valued numbers having integer values greater than zero.” By identifying this as an equivalence class, weare stating that if no error is found by a test of one element of the set,it is unlikely that an error would be found by a test of another element of the set.

In other words, our testing time is best spent elsewhere (in different equivalence classes).Test-case design by equivalence partitioning proceeds in two steps:(1) identifying the equivalence classes and (2) defining the test cases.Identifying the Equivalence ClassesThe equivalence classes are identified by taking each input condition(usually a sentence or phrase in the specification) and partitioning itinto two or more groups.

You can use the table in Figure 4.3 to do54The Art of Software TestingFigure 4.3A form for enumerating equivalence classes.ExternalconditionValid equivalenceclassesInvalid equivalenceclassesthis. Notice that two types of equivalence classes are identified: validequivalence classes represent valid inputs to the program, and invalidequivalence classes represent all other possible states of the condition(i.e., erroneous input values). Thus, we are adhering to the principlediscussed in Chapter 2 that stated that you must focus attention oninvalid or unexpected conditions.Given an input or external condition, identifying the equivalenceclasses is largely a heuristic process.

A set of guidelines is as follows:1. If an input condition specifies a range of values (for example,“the item count can be from 1 to 999”), identify one validequivalence class (1 < item count < 999) and two invalidequivalence classes (item count < 1 and item count > 999).2. If an input condition specifies the number of values (forexample, “one through six owners can be listed for the automobile”), identify one valid equivalence class and two invalidequivalence classes (no owners and more than six owners).3.