The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 6

This attitude began to change in the early1970s through the efforts of program developers who first saw thevalue in reading code as part of a comprehensive testing and debugging regimen.Today, not all testers of software applications read code, but theconcept of studying program code as part of a testing effort certainlyis widely accepted.

Several factors may affect the likelihood that agiven testing and debugging effort will include people actually reading program code: the size or complexity of the application, the sizeof the development team, the timeline for application development(whether the schedule is relaxed or intense, for example), and, ofcourse, the background and culture of the programming team.For these reasons, we will discuss the process of non-computerbased testing (“human testing”), before we delve into the more traditional computer-based testing techniques. Human testing techniquesare quite effective in finding errors—so much so that every programming project should use one or more of these techniques. You shouldapply these methods between the time the program is coded and thetime when computer-based testing begins.

You also can develop andapply analogous methods at earlier stages in the programming process2122The Art of Software Testing(such as at the end of each design stage), but these are outside thescope of this book.But before we begin the discussion of human testing techniques,here’s an important note: Because the involvement of humans resultsin less formal methods than mathematical proofs conducted by acomputer, you may feel skeptical that something so simple and informal can be useful. Just the opposite is true. These informal techniques don’t get in the way of successful testing; rather, theysubstantially contribute to productivity and reliability in two majorways.First, it is generally recognized that the earlier errors are found, thelower the costs of correcting the errors and the higher the probability of correcting the errors correctly. Second, programmers seem toexperience a psychological change when computer-based testingcommences.

Internally induced pressures seem to build rapidly andthere is a tendency to want to “fix this darn bug as soon as possible.”Because of these pressures, programmers tend to make more mistakeswhen correcting an error found during computer-based testing thanthey make when correcting an error found earlier.Inspections and WalkthroughsThe two primary human testing methods are code inspections andwalkthroughs.

Since the two methods have a lot in common, we willdiscuss their similarities together here. Their differences are discussedin subsequent sections.Inspections and walkthroughs involve a team of people reading orvisually inspecting a program. With either method, participants mustconduct some preparatory work. The climax is a “meeting of theminds,” at a participant conference. The objective of the meeting isto find errors but not to find solutions to the errors.

That is, to test,not debug.Code inspections and walkthroughs have been widely used forsome time. In our opinion, the reason for their success is related tosome of the principles in Chapter 2.Program Inspections, Walkthroughs, and Reviews23In a walkthrough, a group of developers—with three or four beingan optimal number—performs the review. Only one of the participants is the author of the program. Therefore, the majority of program testing is conducted by people other than the author, whichfollows the testing principle stating that an individual is usually ineffective in testing his or her own program.An inspection or walkthrough is an improvement over the olderdesk-checking process (the process of a programmer reading his orher own program before testing it). Inspections and walkthroughs aremore effective, again because people other than the program’s authorare involved in the process.Another advantage of walkthroughs, resulting in lower debugging(error-correction) costs, is the fact that when an error is found it isusually precisely located in the code.

In addition, this process frequently exposes a batch of errors, allowing the errors to be correctedlater en masse. Computer-based testing, on the other hand, normallyexposes only a symptom of the error (the program does not terminate or the program prints a meaningless result), and errors are usually detected and corrected one by one.These methods generally are effective in finding from 30 to 70percent of the logic-design and coding errors in typical programs.They are not effective, however, in detecting high-level design errors,such as errors made in the requirements-analysis process.

Note that asuccess rate of 30 to 70 percent doesn’t mean that up to 70 percentof all errors might be found. Remember that Chapter 2 tells us wecan never know the total number of errors in a program. Rather, thismeans these methods are effective in finding up to 70 percent of allerrors found by the end of the testing process.Of course, a possible criticism of these statistics is that the humanprocesses find only the “easy” errors (those that would be trivial tofind with computer-based testing) and that the difficult, obscure, ortricky errors can be found only by computer-based testing.

However,some testers using these techniques have found that the humanprocesses tend to be more effective than the computer-based testingprocesses in finding certain types of errors, while the opposite is true forother types of errors. The implication is that inspections/walkthroughs24The Art of Software Testingand computer-based testing are complementary; error-detection efficiency will suffer if one or the other is not present.Finally, although these processes are invaluable for testing new programs, they are of equal, or even higher, value in testing modifications to programs. In our experience, modifying an existing programis a process that is more error prone (in terms of errors per statementwritten) than writing a new program.

Therefore, program modifications also should be subjected to these testing processes as well asregression testing techniques.Code InspectionsA code inspection is a set of procedures and error-detection techniquesfor group code reading. Most discussions of code inspections focus onthe procedures, forms to be filled out, and so on; here, after a shortsummary of the general procedure, we will focus on the actual errordetection techniques.An inspection team usually consists of four people. One of thefour people plays the role of moderator.

The moderator is expectedto be a competent programmer, but he or she is not the author of theprogram and need not be acquainted with the details of the program.The duties of the moderator include• Distributing materials for, and scheduling, the inspectionsession• Leading the session• Recording all errors found• Ensuring that the errors are subsequently correctedThe moderator is like a quality-control engineer. The second teammember is the programmer.

The remaining team members usuallyare the program’s designer (if different from the programmer) and atest specialist.The moderator distributes the program’s listing and design specifi-Program Inspections, Walkthroughs, and Reviews25cation to the other participants several days in advance of the inspection session. The participants are expected to familiarize themselveswith the material prior to the session. During the session, two activities occur:1. The programmer narrates, statement by statement, the logicof the program. During the discourse, other participantsshould raise questions, and they should be pursued to determine whether errors exist.

It is likely that the programmerrather than the other team members will find many of theerrors found during this narration. In other words, the simpleact of reading aloud a program to an audience seems to be aremarkably effective error-detection technique.2. The program is analyzed with respect to a checklist of historically common programming errors (such a checklist is discussed in the next section).The moderator is responsible for ensuring that the discussions proceed along productive lines and that the participants focus theirattention on finding errors, not correcting them. (The programmercorrects errors after the inspection session.)After the session, the programmer is given a list of the errorsfound. If more than a few errors were found, or if any of the errorsrequires a substantial correction, the moderator might make arrangements to reinspect the program after the errors are corrected.