The art of software testing. Myers (2nd edition) (2004) (811502), страница 25
Текст из файла (страница 25)
Downtime often causes a company to lose revenue because the system is inoperable. One testing objective is toshow that the system fails to meet the service-level agreement forTable 6.1Hours per Year for Various Uptime RequirementsUptime PercentRequirements10099.998979695Operational Hours per Year8760.08751.28584.88497.28409.68322.0142The Art of Software TestingMTTR. Often, the MTTR will have an upper and lower boundary,so your test cases should reflect these bounds.Serviceability TestingThe program also may have objectives for its serviceability or maintainability characteristics. All objectives of this sort must be tested.Such objectives might define the service aids to be provided with thesystem, including storage dump programs or diagnostics, the meantime to debug an apparent problem, the maintenance procedures,and the quality of internal logic documentation.Documentation TestingAs we illustrated in Figure 6.4, the system test also is concerned withthe accuracy of the user documentation.
The principle way ofaccomplishing this is to use the documentation to determine the representation of the prior system test cases. That is, once a particularstress case is devised, you would use the documentation as a guide forwriting the actual test case. Also, the user documentation should bethe subject of an inspection (similar to the concept of the codeinspection in Chapter 3), checking it for accuracy and clarity. Anyexamples illustrated in the documentation should be encoded intotest cases and fed to the program.Procedure TestingFinally, many programs are parts of larger, not completely automatedsystems involving procedures people perform. Any prescribed humanprocedures, such as procedures for the system operator, databaseadministrator, or end user, should be tested during the system test.For example, a database administrator should document procedures for backing up and recovering the database system. If possible,a person not associated with the administration of the databaseshould test the procedures.
However, a company must create theHigher-Order Testing143resources needed to adequately test the procedures. These resourcesoften include hardware and additional software licensing.Performing the System TestOne of the most vital considerations in implementing the system testis determining who should do it. To answer this in a negative way,(1) programmers shouldn’t perform a system test, and (2) of all thetesting phases, this is the one that the organization responsible fordeveloping the programs definitely should not perform.The first point stems from the fact that a person performing a system test must be capable of thinking like an end user, which impliesa thorough understanding of the attitudes and environment of theend user and of how the program will be used. Obviously, then, iffeasible, a good testing candidate is one or more end users.
However,because the typical end user will not have the ability or expertise toperform many of the categories of tests described earlier, an ideal system test team might be composed of a few professional system testexperts (people who spend their lives performing system tests), a representative end user or two, a human-factors engineer, and the keyoriginal analysts or designers of the program. Including the originaldesigners does not violate the earlier principle recommending againsttesting your own program, since the program has probably passedthrough many hands since it was conceived.
Therefore, the originaldesigners do not have the troublesome psychological ties to the program that motivated this principle.The second point stems from the fact that a system test is an “anything goes, no holds barred” activity. Again, the development organization has psychological ties to the program that are counter to thistype of activity. Also, most development organizations are most interested in having the system test proceed as smoothly as possible and onschedule, and are not truly motivated to demonstrate that the program does not meet its objectives.
At the least, the system test shouldbe performed by an independent group of people with few, if any,ties to the development organization. Perhaps the most economical144The Art of Software Testingway of conducting a system test (economical in terms of finding themost errors with a given amount of money, or spending less moneyto find the same number of errors), is to subcontract the test to a separate company. This is discussed further in the last section of thischapter.Acceptance TestingReturning to the overall model of the development process shown inFigure 6.3 on page 127, you can see that acceptance testing is theprocess of comparing the program to its initial requirements and thecurrent needs of its end users. It is an unusual type of test in that itusually is performed by the program’s customer or end user and normally is not considered the responsibility of the development organization. In the case of a contracted program, the contracting (user)organization performs the acceptance test by comparing the program’s operation to the original contract.
As is the case for othertypes of testing, the best way to do this is to devise test cases thatattempt to show that the program does not meet the contract; if thesetest cases are unsuccessful, the program is accepted. In the case of aprogram product, such as a computer manufacturer’s operating system or compiler, or a software company’s database management system, the sensible customer first performs an acceptance test todetermine whether the product satisfies its needs.Installation TestingThe remaining testing process in Figure 6.3 is the installation test. Itsposition in Figure 6.3 is a bit unusual, since it is not related, as all ofthe other testing processes are, to specific phases in the designprocess. It is an unusual type of testing because its purpose is not tofind software errors but to find errors that occur during the installation process.Higher-Order Testing145Many events occur when installing software systems.
A short list ofexamples includes the following:••••User must select a variety of options.Files and libraries must be allocated and loaded.Valid hardware configurations must be present.Programs may need network connectivity to connect to otherprograms.Installation tests should be developed by the organization that produced the system, delivered as part of the system, and run after thesystem is installed. Among other things, the test cases might check toensure that a compatible set of options has been selected, that all partsof the system exist, that all files have been created and have the necessary contents, and that the hardware configuration is appropriate.Test Planning and ControlIf you consider that the testing of a large system could entail writing,executing, and verifying tens of thousands of test cases, handlingthousands of modules, repairing thousands of errors, and employinghundreds of people over a time span of a year or more, it is apparentthat you are faced with an immense project management challenge inplanning, monitoring, and controlling the testing process.
In fact, theproblem is so enormous that we could devote an entire book to justthe management of software testing. The intent of this section is tosummarize some of these considerations.As mentioned in Chapter 2, the major mistake most often made inplanning a testing process is the tacit assumption that no errors willbe found. The obvious result of this mistake is that the plannedresources (people, calendar time, and computer time) will be grosslyunderestimated, a notorious problem in the computing industry.Compounding the problem is the fact that the testing process falls atthe end of the development cycle, meaning that resource changes are146The Art of Software Testingdifficult. A second, perhaps more significant problem is that thewrong definition of testing is being used, since it is difficult to seehow someone using the correct definition of testing (the goal beingto find errors) would plan a test using the assumption that no errorswill be found.As is the case for most undertakings, the plan is the crucial part ofthe management of the testing process.
The components of a goodtest plan are as follows:1. Objectives. The objectives of each testing phase must bedefined.2. Completion criteria. Criteria must be designed to specify wheneach testing phase will be judged to be complete. This matteris discussed in the next section.3. Schedules. Calendar time schedules are needed for each phase.They should indicate when test cases will be designed, written,and executed. Some software methodologies such as ExtremeProgramming (discussed in Chapter 8) require that you designthe test cases and unit tests before application coding begins.4.
Responsibilities. For each phase, the people who will design,write, execute, and verify test cases, and the people who willrepair discovered errors, should be identified. Since in largeprojects disputes unfortunately arise over whether particulartest results represent errors, an arbitrator should be identified.5. Test case libraries and standards. In a large project, systematicmethods of identifying, writing, and storing test cases arenecessary.6.
Tools. The required test tools must be identified, including aplan for who will develop or acquire them, how they will beused, and when they are needed.7. Computer time. This is a plan for the amount of computertime needed for each testing phase. This would includeservers used for compiling applications, if required; desktopmachines required for installation testing; Web servers forWeb-based applications; networked devices, if required; andso forth.Higher-Order Testing1478. Hardware configuration. If special hardware configurations ordevices are needed, a plan is required that describes therequirements, how they will be met, and when they areneeded.9.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
