The art of software testing. Myers (2nd edition) (2004) (811502), страница 22
Текст из файла (страница 22)
Use the code inspection and walkthrough methods discussed in Chapter 3 in the verification step at the end of the seventhprocess.Higher-Order Testing127The third approach is to orient distinct testing processes towarddistinct development processes. That is, focus each testing process ona particular translation step, thus focusing it on a particular class oferrors. This approach is illustrated in Figure 6.3. The testing cycle hasbeen structured to model the development cycle. In other words, youFigure 6.3The correspondence between developmentand testing processes.128The Art of Software Testingshould be able to establish a one-to-one correspondence betweendevelopment and testing processes.
For instance:• The purpose of a module test is to find discrepancies betweenthe program’s modules and their interface specifications.• The purpose of a function test is to show that a program does notmatch its external specifications.• The purpose of a system test is to show that the product isinconsistent with its original objectives.The advantages of this structure are that it avoids unproductiveredundant testing and prevents you from overlooking large classes oferrors. For instance, rather than simply labeling system testing as “thetesting of the whole system” and possibly repeating earlier tests, system testing is oriented toward a distinct class of errors (those madeduring the translation of the objectives to the external specification)and measured with respect to a distinct type of documentation in thedevelopment process.The higher-order testing methods shown in Figure 6.3 are mostapplicable to software products (programs written as a result of a contract or programs intended for wide usage, as opposed to experimentalprograms or programs written for use only by the program’s author).Programs not written as products often do not have formal requirements and objectives; for such programs, the function test might be theonly higher-order test.
Also, the need for higher-order testing increasesas the size of the program increases. The reason is that the ratio of designerrors (errors made in the earlier development processes) to codingerrors is considerably higher in large programs than in small programs.Note that the sequence of testing processes in Figure 6.3 does notnecessarily imply a time sequence. For instance, since system testingis not defined as “the kind of testing you do after function testing,”but instead is defined as a distinct type of testing focused on a distinctclass of errors, it could very well be partially overlapped in time withother testing processes.In this chapter, we discuss the processes of function, system, acceptance, and installation testing. Integration testing is omitted because itHigher-Order Testing129is often not regarded as a separate testing step and, when incrementalmodule testing is used, it is an implicit part of the module test.The discussions of these testing processes will be brief, general,and, for the most part, without examples because specific techniquesused in these higher-order tests are highly dependent on the specificprogram being tested.
For instance, the characteristics of a system test(the types of test cases, the manner in which test cases are designed,the test tools used) for an operating system will differ considerablyfrom a system test of a compiler, a program controlling a nuclearreactor, or a database application program.The last few sections in this chapter discuss planning and organizational issues and the important question of determining when to stoptesting.Function TestingAs indicated in Figure 6.3, function testing is a process of attemptingto find discrepancies between the program and the external specification.
An external specification is a precise description of the program’s behavior from the point of view of the end user.Except when used on small programs, function testing is normallya black-box activity. That is, you rely on the earlier module-testingprocess to achieve the desired white-box logic-coverage criteria.To perform a function test, the specification is analyzed to derive aset of test cases. The equivalence-partitioning, boundary-value analysis, cause-effect graphing, and error-guessing methods described inChapter 4 are especially pertinent to function testing.
In fact, theexamples in Chapter 4 are examples of function tests. The descriptions of the FORTRAN DIMENSION statement, the examinationscoring program, and the DISPLAY command actually are examples ofexternal specifications. (Note, however, that they are not completelyrealistic examples; for instance, a real external specification for thescoring program would include a precise description of the format ofthe reports.) Hence, no examples of function tests are presented in thissection.130The Art of Software TestingMany of the guidelines of Chapter 2 also are particularly pertinentto function testing. Keep track of which functions have exhibited thegreatest number of errors; this information is valuable because it tellsus that these functions probably also contain the preponderance of asyet undetected errors.
Remember to focus a sufficient amount ofattention on invalid and unexpected input conditions. Recall that thedefinition of the expected result is a vital part of a test case. Finally, asalways, remember that the purpose of the function test is to exposeerrors and discrepancies with the specification, not to demonstratethat the program matches its external specification.System TestingSystem testing is the most misunderstood and most difficult testingprocess. System testing is not a process of testing the functions of thecomplete system or program, because this would be redundant withthe process of function testing. As shown in Figure 6.3, system testing has a particular purpose: to compare the system or program to itsoriginal objectives.
Given this purpose, two implications are as follows:1. System testing is not limited to systems. If the product is aprogram, system testing is the process of attempting todemonstrate how the program, as a whole, does not meet itsobjectives.2. System testing, by definition, is impossible if there is no set ofwritten, measurable objectives for the product.In looking for discrepancies between the program and its objectives, focus on translation errors made during the process of designing the external specification. This makes the system test a vital testprocess, because in terms of the product, the number of errors made,and the severity of those errors, this step in the development cycleusually is the most error prone.It also implies that, unlike the function test, the external specifica-Higher-Order Testing131tion cannot be used as the basis for deriving the system test cases,since this would subvert the purpose of the system test.
On the otherhand, the objectives document cannot be used, by itself, to formulatetest cases, since it does not, by definition, contain precise descriptionsof the program’s external interfaces. We solve this dilemma by usingthe program’s user documentation or publications. Design the systemtest by analyzing the objectives; formulate test cases by analyzing theuser documentation. This has the useful side effect of comparing theprogram to its objectives and to the user documentation, as well ascomparing the user documentation to the objectives, as shown inFigure 6.4Figure 6.4 illustrates why system testing is the most difficult testingprocess. The leftmost arrow in the figure, comparing the program toFigure 6.4The system test.132The Art of Software Testingits objectives, is the central purpose of the system test, but there areno known test-case-design methodologies. The reason for this is thatobjectives state what a program should do and how well the programshould do it, but they do not state the representation of the program’sfunctions.
For instance, the objectives for the DISPLAY commandspecified in Chapter 4 might have read as follows:A command will be provided to view, from a terminal, the contentsof main-storage locations. Its syntax should be consistent with thesyntax of all other system commands. The user should be able tospecify a range of locations, via an address range or an address and acount.
Sensible defaults should be provided for command operands.Output should be displayed as multiple lines of multiple words(in hexadecimal), with spacing between the words. Each line shouldcontain the address of the first word of that line. The command is a“trivial” command, meaning that under reasonable system loads, itshould begin displaying output within two seconds, and there shouldbe no observable delay time between output lines. A programmingerror in the command processor should, at the worst, cause thecommand to fail; the system and the user’s session must not beaffected. The command processor should have no more than oneuser-detected error after the system is put into production.Given the statement of objectives, there is no identifiable methodology that would yield a set of test cases, other than the vague butuseful guideline of writing test cases to attempt to show that the program is inconsistent with each sentence in the objectives statement.Hence, a different approach to test-case design is taken here; ratherthan describing a methodology, distinct categories of system test casesare discussed.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
