The art of software testing. Myers (2nd edition) (2004) (811502), страница 23
Текст из файла (страница 23)
Because of the absence of a methodology, system testing requires a substantial amount of creativity; in fact, the design ofgood system test cases requires more creativity, intelligence, andexperience than are required to design the system or program.The 15 categories of test cases are discussed here. We don’t claimthat all 15 categories will be applicable to every program, but, toHigher-Order Testing133avoid overlooking something, all 15 categories should be exploredwhen designing test cases.Facility TestingThe most obvious type of system testing is the determination ofwhether each facility (or function, but the word “function” is notused here to avoid confusing this with function testing) mentioned inthe objectives was actually implemented. The procedure is to scanthe objectives sentence by sentence and, when a sentence specifies awhat (for example, “syntax should be consistent .
. . ,” “user shouldbe able to specify a range of locations . . .”), determine that the program satisfies the “what.” This type of testing often can be performed without a computer; a mental comparison of the objectiveswith the user documentation is sometimes sufficient. Nonetheless, achecklist is helpful to ensure that you mentally check the same objectives the next time you perform the test.Volume TestingA second type of system testing is subjecting the program to heavyvolumes of data. For instance, a compiler would be fed an absurdlylarge source program to compile. A linkage editor might be fed aprogram containing thousands of modules.
An electronic circuit simulator would be given a circuit containing thousands of components.An operating system’s job queue would be filled to capacity. If a program is supposed to handle files spanning multiple volumes, enoughdata is created to cause the program to switch from one volume toanother. In other words, the purpose of volume testing is to showthat the program cannot handle the volume of data specified in itsobjectives.Since volume testing obviously can require significant resources, interms of machine and people time, you can’t go overboard.
However, every program must be exposed to at least a few volume tests.134The Art of Software TestingStress TestingStress testing subjects the program to heavy loads or stresses. Thisshould not be confused with volume testing; a heavy stress is a peakvolume of data, or activity, encountered over a short span of time. Ananalogy would be evaluating a typist. A volume test would determinewhether the typist could cope with a draft of a large report; a stresstest would determine whether the typist could type at a rate of 50words per minute.Because stress testing involves an element of time, it is not applicable to many programs, for example, a compiler or a batch-processingpayroll program.
It is applicable, however, to programs that operateunder varying loads, or interactive, real-time, and process controlprograms. If an air traffic control system is supposed to keep track ofup to 200 planes in its sector, you could stress-test it by simulating theexistence of 200 planes. Since there is nothing to physically keep a201st plane from entering the sector, a further stress test wouldexplore the system’s reaction to this unexpected plane. An additionalstress test might simulate the simultaneous entry of a large number ofplanes into the sector.If an operating system is supposed to support a maximum of 15multiprogrammed jobs, the system could be stressed by attempting torun 15 jobs simultaneously. Stress a pilot training aircraft simulator bydetermining the system’s reaction to the trainee’s forcing the rudderleft, pulling back on the throttle, lowering the flaps, lifting the nose,lowering the landing gear, turning on the landing lights, and bankingleft, all at the same time.
(Such test cases might require a four-handedpilot or, realistically, two test specialists in the cockpit.) You mightstress-test a process control system by causing all of the monitoredprocesses to generate signals simultaneously. A telephone switchingsystem is stressed by routing to it a large number of simultaneousphone calls.One of the common recipients of stress testing is Web-based applications. Here you want to ensure that your application, and hardware, can handle some volume of concurrent users. You could arguethat you may have millions of people accessing the site at one time,Higher-Order Testing135but that is not realistic.
You need to understand your audience, thendesign a stress test to represent the maximum number of users youthink will use your site. Chapter 9 provides more information ontesting Web-based applications.Although many stress tests do represent conditions that the program likely will experience during its operation, other stress testsmay truly represent “never will occur” situations, but this does notimply that these tests are not useful. If these impossible conditionsdetect errors, the test is valuable because it is likely that the sameerrors might also occur in realistic, less stressful situations.Usability TestingAnother important category of system test cases is an attempt to findhuman-factor, or usability, problems.
When the first edition of thisbook was published, the computing industry had given very littleattention to studying and defining good human-factor considerationsof programming systems. Today’s software systems—particularlythose designed for a mass, commercial market—generally haveundergone extensive human-factor studies, and modern programs, ofcourse, benefit from the thousands of programs and systems that havegone before.
Nevertheless, an analysis of human factors is still ahighly subjective matter. Here’s our list of considerations that mightbe tested:1. Has each user interface been tailored to the intelligence, educational background, and environmental pressures of the enduser?2. Are the outputs of the program meaningful, nonabusive, anddevoid of computer gibberish?3. Are the error diagnostics, such as error messages, straightforward, or does the user need a PhD in computer science tocomprehend them? For instance, does the program producesuch messages as “IEK022A OPEN ERROR ON FILE‘SYSIN’ ABEND CODE=102?” Messages such as theseweren’t all that uncommon in software systems of the 1970s136The Art of Software Testingand 1980s. Mass-market systems do better today, but you stillwill encounter unhelpful messages such as “An unknownerror has occurred” or “This program has encountered anerror and must be restarted.” Programs you design yourselfare under your control and should not be plagued with suchuseless messages.
Even if you didn’t design the program, ifyou are on the testing team you can push for improvementsin this area of the human interface.4. Does the total set of user interfaces exhibit considerable conceptual integrity, an underlying consistency, and uniformityof syntax, conventions, semantics, format, style, and abbreviations?5.
Where accuracy is vital, such as in an online banking system,is sufficient redundancy present in the input? For example,such a system should ask for an account number, a customername, and a PIN (personal identification number) to verifythat the proper person is accessing account information.6. Does the system contain an excessive number of options, oroptions that are unlikely to be used? One trend in modernsoftware is to present to the user only those menu choices theyare most likely to use, based on software testing and designconsiderations.
Then a well-designed program can learn fromthe user and begin to present those menu items that individualusers frequently access. Even with such an intelligent menusystem, successful programs still must be designed so thataccessing the various options is logical and intuitive.7. Does the system return some type of immediate acknowledgment to all inputs? Where a mouse click is the input, forexample, the chosen item can change color or a buttonobject can depress or be presented in a raised format.
If theuser is expected to choose from a list, the selected numbershould be presented on the screen when the choice is made.Moreover, if the selected action requires some processingtime—which is frequently the case if the software is accessinga remote system—then a message should be displayedinforming the user of what is going on.Higher-Order Testing1378. Is the program easy to use? For example, is the input casesensitive without making this fact clear to the user? Also, if aprogram requires navigation through a series of menus oroptions, is it clear how to return to the main menu? Can theuser easily move up or down one level?Security TestingBecause of society’s increasing concern about privacy, many programshave specific security objectives.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
