The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 21

Also, because upper levelsoften provide resources for use by lower levels (e.g., opening of files),it is difficult sometimes to determine whether the resources havebeen provided correctly (e.g., if a file has been opened with theproper attributes) until the lower modules that use them are tested.Bottom-up TestingThe next step is to examine the bottom-up incremental testingstrategy. For the most part, bottom-up testing is the opposite oftop-down testing; the advantages of top-down testing become thedisadvantages of bottom-up testing, and the disadvantages of topdown testing become the advantages of bottom-up testing. Becauseof this, the discussion of bottom-up testing is shorter.The bottom-up strategy begins with the terminal modules in theprogram (the modules that do not call other modules). After theseModule (Unit) Testing117modules have been tested, again there is no best procedure for selecting the next module to be incrementally tested; the only rule is that,to be eligible to be the next module, all of the module’s subordinatemodules (the modules it calls) must have been tested previously.Returning to Figure 5.8, the first step is to test some or all of modules E, J, G, K, L, and I, either serially or in parallel.

To do so, eachmodule needs a special driver module: a module that contains wiredin test inputs, calls the module being tested, and displays the outputs(or compares the actual outputs with the expected outputs). Unlikethe situation with stubs, multiple versions of a driver are not needed,since the driver module can iteratively call the module being tested. Inmost cases, driver modules are easier to produce than stub modules.As was the case earlier, a factor influencing the sequence of testingis the critical nature of the modules. If we decide that modules D andF are most critical, an intermediate state of the bottom-up incremental test might be that of Figure 5.11.

The next steps might be totest E and then test B, combining B with the previously tested modules E, F, and J.A drawback of the bottom-up strategy is that there is no conceptof an early skeletal program. In fact, the working program does notexist until the last module (module A) is added, and this workingprogram is the complete program. Although the I/O functions canbe tested before the whole program has been integrated (the I/Omodules are being used in Figure 5.11), the advantages of the earlyskeletal program are not present.The problems associated with the impossibility or difficulty of creating all test situations in the top-down approach do not exist here.

Ifyou think of a driver module as a test probe, the probe is being placeddirectly on the module being tested; there are no intervening modules to worry about. Examining other problems associated with thetop-down approach, you can’t make the unwise decision to overlapdesign and testing, since the bottom-up test cannot begin until thebottom of the program has been designed.

Also, the problem of notcompleting the test of a module before starting another, because ofthe difficulty of encoding test data in versions of a stub, does not existwhen using bottom-up testing.118The Art of Software TestingFigure 5.11Intermediate state in the bottom-up test.A ComparisonIt would be convenient if the top-down versus bottom-up issue wereas clear-cut as the incremental versus nonincremental issue, butunfortunately it is not.

Table 5.3 summarizes their relative advantagesand disadvantages (excluding the previously discussed advantagesshared by both—the advantages of incremental testing). The firstadvantage of each approach might appear to be the deciding factor,but there is no evidence showing that major flaws occur more oftenat the top or bottom levels of the typical program.

The safest way tomake a decision is to weigh the factors in Table 5.3 with respect tothe particular program being tested. Lacking such a program here,the serious consequences of the fourth disadvantage of top-downtesting and the availability of test tools that eliminate the need fordrivers but not stubs seem to give the bottom-up strategy the edge.In addition, it may be apparent that top-down and bottom-up testing are not the only possible incremental strategies.Table 5.3Comparison of Top-down and Bottom-up TestingTop-down TestingAdvantages1. This is advantageous if majorflaws occur toward the topof the program.2.

Once the I/O functions areadded, representation of testcases is easier.3. Early skeletal program allowsdemonstrations and boostsmorale.Disadvantages1. Stub modules must beproduced.2. Stub modules are often morecomplicated than they firstappear to be.3. Before the I/O functions areadded, the representation oftest cases in stubs can bedifficult.4. Test conditions may beimpossible, or very difficult, tocreate.5. Observation of test output ismore difficult.6.

It allows one to think thatdesign and testing can beoverlapped.7. It induces one to defercompletion of the testing ofcertain modules.Bottom-up TestingAdvantages1. This is advantageous if majorflaws occur toward the bottomof the program.2. Test conditions are easier tocreate.3. Observation of test results iseasier.Disadvantages1. Driver modules must beproduced.2. The program as an entity doesnot exist until the last moduleis added.119120The Art of Software TestingPerforming the TestThe remaining part of the module test is the act of actually carryingout the test.

A set of hints and guidelines for doing this are describedhere.When a test case produces a situation where the module’s actualresults do not match the expected results, there are two possibleexplanations: either the module contains an error or the expectedresults are incorrect (the test case is incorrect). To minimize this confusion, the set of test cases should be reviewed or inspected before thetest is performed (that is, the test cases should be tested).The use of automated test tools can minimize part of the drudgeryof the testing process. For instance, there exist test tools that eliminate the need for driver modules.

Flow-analysis tools enumerate thepaths through a program, find statements that can never be executed(“unreachable” code), and find instances where a variable is usedbefore it is assigned a value.It is helpful, when preparing for a module test, to review the psychological and economic principles discussed in Chapter 2.

As wasthe practice earlier in this chapter, remember that a definition of theexpected result is a necessary part of a test case. When executing atest, remember to look for side effects (instances where a moduledoes something it is not supposed to do). In general, these situationsare difficult to detect, but some instances may be found by checking,after execution of the test case, the inputs to the module that are notsupposed to be altered.

For instance, test case 7 in Figure 5.6 statesthat, as part of the expected result, ESIZE, DSIZE, and DEPTTAB shouldbe unchanged. When running this test case, not only is the outputexamined for the correct result, but ESIZE, DSIZE, and DEPTTAB shouldbe examined to determine whether they were erroneously altered.The psychological problems associated with a person attemptingto test his or her own programs apply to module testing. Rather thantesting their own modules, programmers might swap modules; theprogrammer of the calling module is always a good candidate to testthe called module. Note that this applies only to testing; the debugging of a module always should be performed by the original pro-Module (Unit) Testing121grammer.

Avoid throwaway test cases; represent them in such a formthat they can be reused in the future. Recall the counterintuitivephenomenon in Figure 2.2. If an abnormally high number of errorsare found in a subset of the modules, it is likely that these modulescontain even more, as yet undetected, errors. Such modules shouldbe subjected to further module testing, and possibly an additionalcode walkthrough or inspection. Finally, remember that the purposeof a module test is not to demonstrate that the module functions correctly, but to demonstrate the presence of errors in the module.CHAPTER 6Higher-Order TestingWhen you finish module-testing aprogram, you have really only just begun the testing process. This isespecially true of large or complex programs.

Consider this important concept:A software error occurs when the program does not dowhat its end user reasonably expects it to do.Applying this definition, even if you could perform an absolutelyperfect module test, you still couldn’t guarantee that you have foundall software errors.To complete testing, then, some form of further testing is necessary. We call this new form higher-order testing.Software development is largely a process of communicating information about the eventual program and translating this informationfrom one form to another.

For that reason, the vast majority of software errors can be attributed to breakdowns, mistakes, and noise during the communication and translation of information.This view of software development is illustrated in Figure 6.1, amodel of the development cycle for a software product. The flow ofthe process can be summarized in seven steps:1. The program user’s needs are translated into a set of writtenrequirements.

These are the goals for the product.2. The requirements are translated into specific objectives byassessing feasibility and cost, resolving conflicting requirements, and establishing priorities and trade-offs.123124The Art of Software TestingFigure 6.1The software development process.3. The objectives are translated into a precise product specification, viewing the product as a black box and consideringonly its interfaces and interactions with the end user. Thisdescription is called the external specification.4. If the product is a system such as an operating system,flight-control system, database management system, orHigher-Order Testing125employee personnel system, rather than a program (compiler, payroll program, word processor), the next process issystem design. This step partitions the system into individual programs, components, or subsystems, and defines theirinterfaces.5.

The structure of the program or programs is designed byspecifying the function of each module, the hierarchicalstructure of the modules, and the interfaces between modules.6. A precise specification is developed that defines the interfaceto, and function of, each module.7. Through one or more substeps, the module interface specification is translated into the source-code algorithm of eachmodule.Here’s another way of looking at these forms of documentation:• Requirements specify why the program is needed.• Objectives specify what the program should do and how wellthe program should do it.• External specifications define the exact representation of theprogram to users.• Documentation associated with the subsequent processesspecifies, in increasing levels of detail, how the program isconstructed.Given the premise that the seven steps of the development cycleinvolve communication, comprehension, and translation of information, and the premise that most software errors stem from breakdowns in information handling, there are three complementaryapproaches to prevent and/or detect these errors.

First, we can introduce more precision into the development process to prevent manyof the errors. Second, we can introduce, at the end of each process, aseparate verification step to locate as many errors as possible beforeproceeding to the next process. This approach is illustrated in Figure6.2. For instance, the external specification is verified by comparing126The Art of Software TestingFigure 6.2The development process withintermediate verification steps.it to the output of the prior stage (the statement of objectives) andfeeding back any discovered mistakes to the external-specificationprocess.